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Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely  oversight 
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Defense  and  Congress;  and  informs  the  public. 
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Our  vision  is  to  be  a  model  oversight  organization  in  the  Federal 
Government  by  leading  change,  speaking  truth,  and  promoting 
excellence — a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 


Fraud,  Waste  &  Abuse 

HOTLINE 

Department  of  Defense 

dodig.mil/hotlinel8oo.424.9098 


For  more  information  about  whistleblower  protection,  please  see  the  inside  back  cover. 


FOR  OFFICIAL  USE  ONLY 


run:  im  at  ttce  rwn  v 

Results  in  Brief 
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April  27,  2015 


Findings  (cont'd) 


Objective 

We  inspected  the  F-35  engine  (F135] 
program's  quality  management  system 
for  conformity  to  contractually  required 
AS9100C,  "Quality  Management  System," 
statutory  and  regulatory  requirements, 

DoD  policies,  and  internal  quality  processes 
and  procedures.  F135  engines  are  procured 
by  the  Department  of  Defense  from  Pratt  & 
Whitney  for  the  F-35  Lightning  II  Program. 

Findings 

A.  Additional  program  management 
oversight  is  required  by  the 

F-35  Joint  Program  Office  (JPO)  and 
the  Defense  Contract  Management 
Agency  (DCMA),  as  evidenced  by  the 
61  nonconformities  (violations  of 
AS9100C,  regulatory  requirements, 
and  DoD  policies]  that  we  documented 
during  our  inspection. 

B.  The  F135  critical  safety  item  (CSI) 
program  did  not  meet  DoD  CSI 
requirements,  including  requirements 
for  parts  identification,  critical 
characteristic  identification,  part 
determination  methodology,  and 
supplier  identification. 

C.  (FOUO)  The  F-35  JPO  did  not 
establish  F135  program  quality  goals 
and  objectives  that  were  mutually 
agreed  upon  by  Pratt  &  Whitney 

for  current  contracts.  Additionally, 
Pratt  &  Whitney  metrics  did  not 
show  improvement  in  quality 


assurance,  process  capability,  and 
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D.  The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney 
proactively  identified,  elevated,  tracked,  and  managed 
F135  program  risks,  in  accordance  with  the  F135  risk 
management  plan. 

E.  The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney's 
supplier  selection  criteria  and  management  of 
underperforming  suppliers  were  sufficient. 

F.  The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney 
demonstrated  adequate  software  quality  management 
practices.  Pratt  &  Whitney  had  an  outdated  software 
development  plan,  requirements  traceability  issues,  and 
a  software  quality  assurance  organization  that  did  not 
perform  required  functions. 

Recommendations 

A.l.a  -  A.l.c.  We  recommend  that  the  F-35  JPO  coordinate 
with  DCMA  to  conduct  an  effective  root  cause  analysis  and 
implement  corrective  actions  for  all  61  nonconformities 
identified  during  our  inspection;  review  the  contract  data 
requirements  list  (CDRL]  and  determine  specific  items  that 
should  require  approval;  and  evaluate  open  major  variance 
requests  to  determine  whether  specification  changes  are 
required  and  if  achievable  closure  plans  can  be  developed. 

A.  2. a  -  A.2.C.  We  recommend  that  DCMA  review  CDRL 
items  to  identify  any  deliverables  that  impact  DCMA  program 
surveillance  and  coordinate  with  the  F-35  JPO  to  resolve  CDRL 
distribution  issues;  review  and  update  the  Memorandum 

of  Agreement  (MOA]  with  the  F-35  JPO  to  ensure  that 
all  DCMA  functions  to  evaluate  waivers  and  deviations 
are  clear  and  in  accordance  with  the  Federal  Acquisition 
Regulation;  and  ensure  that  all  associated  data  required  to 
substantiate  corrective  action  closure  are  accessible,  available, 
and  retrievable. 

B. l  -  B.2.  We  recommend  that  the  F-35  JPO  ensure  that 
contractual  flow  down  of  all  CSI  requirements  and  Pratt  & 
Whitney's  CSI  program  processes  and  specifications  meet 
the  intent  of  current  DoD  CSI  requirements  and  ensure  that 
CSI  parts  or  assemblies  for  already  delivered  engines  meet 
current  DoD  CSI  requirements. 

FOR  OFFICIAL  USE  ONLY 

DODIG-2015-111  (Project  No.  D2014-DT0TAD-003.000)  |  i 


run:  im  at  ttce  rwn  v 

Results  in  Brief 

F-35  Engine  Quality  Assurance  Inspection 


Recommendations  (cont'd) 

C. l  -  C.3.  We  recommend  that  the  F-35  JPO  establish 
F135  program  quality  goals  and  objectives  that 

are  mutually  agreed  upon  by  Pratt  &  Whitney  and 
track  Pratt  &  Whitney's  performance  against  those 
objectives;  ensure  that  Pratt  &  Whitney's  quality  plan 
meets  contractual  requirements;  and  ensure  that 
Pratt  &  Whitney  consistently  analyzes  and  reports  key 
product  characteristic  process  capability  index  (Cpk) 
data  for  F135  engine  hardware  and  that  performance 
improvement  plans  are  established. 

D.  We  recommend  that  the  F-35  JPO  ensure  that 
Pratt  &  Whitney  identify,  elevate,  track,  and  manage 
all  risks  that  affect  the  program,  including  software 
and  supply  chain  risks. 

E. l.  We  recommend  that  the  F-35  JPO  work  with 
DCMA  to  ensure  that  Pratt  &  Whitney  clearly  defines, 
documents,  and  implements  minimum  baseline  criteria 
for  supplier  selection  and  actions  to  be  taken  for 
suppliers  that  continue  to  be  high  risk. 

E. 2.  We  recommend  that  DCMA  perform  additional 
surveillance  on  Pratt  &  Whitney's  corrective  action 
requests  issued  to  suppliers  and  ensure  that  they  are 
closed  within  a  reasonable  timeframe. 

F. l  -  F.3.  We  recommend  that  the  F-35  JPO  ensure 
that  Pratt  &  Whitney  resolve  nonconformities 
related  to  software  quality  management  systems; 
ensure  that  Pratt  &  Whitney's  Software  Quality 
Assurance  organization  conduct  audits,  reviews,  and 
verification  activities  of  both  internally  developed 
and  supplier-developed  software;  and  ensure  that 
Pratt  &  Whitney  assess  the  impact  of  insufficient 
software  verification  on  delivered  engines. 


Management  Comments 
and  Our  Response 

The  F-35  JPO  and  DCMA  provided  comments  on 
our  findings  and  recommendations.  The  F-35  JPO 
agreed  with  seven  recommendations,  partially 
agreed  with  two  recommendations,  and  disagreed 
with  four  recommendations.  DCMA  agreed  with 
three  recommendations  and  partially  agreed  with 
one  recommendation. 

The  F-35  JPO  disagreed  with  evaluating  open  variance 
requests,  stating  that  the  F135  program  is  still  in 
development  and  the  F-35  JPO  is  working  to  either  meet 
or  change  the  requirements  at  the  end  of  the  system 
development  and  demonstration  phase.  The  F-35  JPO 
partially  agreed  with  our  recommendations  on  CSIs 
but  still  plans  to  align  with  updated  DoD  requirements 
and  to  work  with  the  contractor  to  align  with  DoD 
requirements  by  May  2015.  For  engines  already 
delivered,  the  F-35  JPO  will  first  determine  if  significant 
escapes  are  discovered  while  aligning  the  F135 
program  with  current  DoD  requirements  before  taking 
appropriate  action.  The  F-35  JPO  partially  agreed  with 
our  recommendation  on  risk  management  but  committed 
to  ensure  that  Pratt  &  Whitney  identifies,  elevates, 
tracks,  and  manages  all  risks  that  affect  the  program. 
The  F-35  JPO  disagreed  with  our  recommendation  on 
software  quality  management,  stating  that  software 
is  developed  to  the  correct  product  software  level,  the 
software  development  plan  is  not  obsolete,  and  the 
F135  propulsion  system  software  was  properly  tested. 
Comments  from  the  F-35  JPO  did  not  fully  address  the 
specifics  of  our  recommendations;  therefore,  further 
comments  are  required. 

DCMA  partially  agreed  with  our  recommendation  to 
document  corrective  action  closures  but  committed  to 
work  on  a  central  repository  to  maintain  corrective 
action  documentation.  Comments  from  DCMA 
addressed  all  specifics  of  the  recommendations,  and 
no  further  comments  are  required.  Please  see  the 
Recommendations  Table  on  the  next  page. 
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Recommendations  Table 


Management 

Recommendations 
Requiring  Comment 

No  Additional 

Comments  Required 

F-35  Joint  Program  Office 

A.l.c,  B.2,  C.l,  F.l,  F.3 

A.l.a-b,  B.l,  C.2,  C.3,  D,  E.l,  F.2 

Defense  Contract  Management  Agency 

A.2.a-c,  E.2 

Please  provide  Management  Comments  by  May  29,  2015. 
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INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 

4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 


April  27,  2015 


MEMORANDUM  FOR  PROGRAM  EXECUTIVE  OFFICER  JOINT  STRIKE  FIGHTER 
DIRECTOR,  DEFENSE  CONTRACT  MANAGEMENT  AGENCY 

SUBJECT:  F-35  Engine  Quality  Assurance  Inspection  (Report  No.  DODIG-2015-111) 

The  DoD  Office  of  Inspector  General  (OIG)  conducted  an  inspection  of  the  F-35  engine  (F135) 
program’s  quality  management  system  for  conformity  to  contractually  required  AS9100C, 
"Quality  Management  System,"  statutory  and  regulatory  requirements,  DoD  policies,  and 
internal  quality  processes  and  procedures.  F135  engines  are  procured  by  the  Department  of 
Defense  from  Pratt  &  Whitney  for  the  F-35  Lightning  II  Program. 

We  conducted  this  evaluation  in  accordance  with  the  Council  of  Inspectors  General  on 
Integrity  and  Efficiency,  "Quality  Standards  for  Inspection  and  Evaluation." 

During  our  inspection,  we  identified  61  nonconformities  that  were  violations  of  AS9100C, 
regulatory  requirements,  and  DoD  policies.  Based  on  those  nonconformities,  we  identified 
systemic  findings  in  the  following  areas:  program  management  oversight,  critical  safety  item 
compliance,  continuous  improvement,  risk  management,  supplier  management,  and  software 
quality  management.  By  implementing  our  recommendations,  we  believe  that  improvements 
can  be  made  to  these  areas. 

We  considered  management  comments  on  the  draft  of  this  report  when  preparing  the  final 
report.  DoD  Instruction  7650.03  requires  that  recommendations  be  resolved  promptly. 
Comments  from  the  F-35  Joint  Program  Office  did  not  fully  address  the  specifics  of  our 
recommendations;  therefore,  we  request  further  comments  on  Recommendations  A.l.c,  B.2, 
C.l,  F.l,  and  F.3  by  May  29,  2015.  Comments  from  the  Defense  Contract  Management  Agency 
were  fully  responsive;  therefore,  no  further  comments  are  required. 

Please  provide  comments  that  conform  to  the  requirements  for  DoD  Instruction  7650.03. 

If  possible,  please  send  a  PDF  file  containing  your  comments  to 

Copies  of  your  comments  must  have  the  actual  signature  of  the  authorizing  official  for 
your  organization.  We  cannot  accept  the  /Signed/  symbol  in  place  of  the  actual  signature. 

If  you  arrange  to  send  classified  comments  electronically,  you  must  send  them  over  the 
SECRET  Internet  Protocol  Router  Network  (SIPRNET). 
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We  appreciate  the  courtesies  extended  to  the  staff.  Please  direct  questions  to 


-''Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 


cc:  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 
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Introduction 


Objectives 

Our  objective  was  to  inspect  the  F-35  engine  (F135)  program’s  quality  management 
system  for  conformity  to  contractually  required  AS9100C,  "Quality  Management 
System,"  statutory  and  regulatory  requirements,  DoD  policies,  and  internal  quality 
processes  and  procedures.  F135  engines  are  procured  by  the  Department  of 
Defense  from  Pratt  &  Whitney  for  the  F-35  Lightning  II  Program. 

Background 

The  F-35  Lightning  II  Program  is  a  joint,  multiservice,  and  multinational  acquisition 
to  develop  and  field  the  next-generation  strike  fighter  aircraft  for  the  Navy, 

Air  Force,  Marine  Corps,  and  several  international  partners  and  foreign  military 
customers.  The  F-35  Joint  Program  Office  (JPO)  is  responsible  for  the  total  lifecycle 
management  of  the  program  and  uses  product-focused  integrated  product  teams  to 
manage  and  execute  program  requirements. 

The  F-35,  a  single  engine  aircraft,  has  three  variants;  F-35A  Conventional  Takeoff 
and  Landing  (CTOL),  F-35B  Short  Takeoff  and  Vertical  Landing  (STOVL),  and 
F-35C  Carrier-Suitable  Variant  (CV).  Lockheed  Martin  is  the  prime  contractor 
for  the  air  vehicle  and  Pratt  &  Whitney  is  the  prime  contractor  for  the 
engine.  Pratt  &  Whitney  produces  engines  for  both  commercial  and  military 
applications.  Their  engines  are  found  in  the  F-15  Eagle,  F-16  Fighting  Falcon, 

F-22  Raptor,  F-35  Lightning  II,  C-17  Globemaster  III  military  transport,  and 
Boeing’s  KC-46  tanker.  The  F135  engine,  which  Pratt  &  Whitney  produces  for  the 
F-35  Lightning  II  fighter  is  the  subject  of  this  report. 

F135  Engine 

There  are  two  engine  types  for  the  F-35  aircraft;  the  F135-PW-100  for  the  CTOL 
and  CV  variants  and  the  F135-PW-600  for  the  STOVL  variant.  The  F135-PW-100 
and  F135-PW-600  engines  are  comprised  of  five  modules:  fan  module,  power 
module,  augmentor  module,  nozzle  module,  and  gearbox  module.  These  modules, 
as  depicted  in  Figure  1,  can  be  removed  and  replaced  during  maintenance  with 
components  and  accessories.  The  F135-PW-600,  as  depicted  in  Figure  2,  has 
additional  components,  including  variable  area  vane  box  nozzle,  lift  fan,  lift 
fan  clutch,  driveshaft,  roll  posts,  and  3-bearing  swivel  duct.  The  two  major 
subcontractors  for  the  F135  engine  are  Rolls-Royce  and  UTC  Aerospace 
Systems  (UTAS).  UTAS  provides  external  controls  and  engine  gearbox. 

Rolls-Royce  provides  the  lift  system  for  the  STOVL  variant. 
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Figure  1.  F135  Engine  Modules 
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Figure  2.  F135-PW-600  Engine  for  the  STOVL  Variant 


Pratt  &  Whitney  manufactures  engine  module  parts  at  its  East  Hartford  and 
Middletown,  Connecticut,  facilities.  The  engine  assembly  and  acceptance  testing 
takes  place  at  its  Middletown,  Connecticut,  and  West  Palm  Beach,  Florida,  facilities. 
The  current  build  time  of  an  engine  is  56  days,  which  includes  approximately 
5  days  of  testing  and  12  days  of  packaging  and  acceptance.  The  F-35  JPO  purchases 
complete  engines  and  spare  parts.  Once  manufactured  and  accepted  by  the 
government,  complete  engines  that  are  slated  for  aircraft  installation  are  provided 
to  Lockheed  Martin  as  Government-furnished  equipment,  and  complete  engine 
spares  and  spare  parts  are  stored  in  depots  throughout  the  country  until  needed. 
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F135  Acquisition  Strategy 

Pratt  &  Whitney  is  executing  a  cost-reimbursement  contract  for  the  development, 
production,  qualification,  delivery,  and  flight  test  support  for  the  system 
development  and  demonstration  (SDD)  phase  of  F135  program.  F135  SDD  contract 
efforts  are  being  performed  concurrently  with  the  execution  of  separate  low-rate 
initial  production  (LRIP)1  lot  deliveries.  Each  LRIP  lot  delivers  an  increasing  level 
of  maturity  as  additional  system  capability  is  achieved.  Each  LRIP  lot  has  its  own 
contract  that  establishes  engine  and  spares  quantities.  The  contracting  approach 
for  LRIP  lots  1  through  3  was  cost-plus  incentive/award/fixed  fee.  The  contracting 
approach  for  LRIP  lots  4  through  10  employs  a  fixed-price-incentive  (firm  target) 
type  contract  for  production.  The  LRIP  and  full  rate  production  (FRP)  contract 
values  and  delivery  schedule  are  in  Table  1. 


Table  1.  Contract  Values  and  Delivery 


Total  Value 

Engine  Quantity 
(installed/spares) 

Scheduled  Delivery 

LRIP  1 

$87,  543,741 

2/2 

FY  2010 

LRIP2 

$508,701,220 

12/4 

FY  2010 

LRIP  3 

$710,813,914 

17/3 

FY  2011 

LRIP  4 

$1,158,185,692 

32/5 

FY  2012 

LRIP  5 

$1,016,351,233 

32/4 

FY  2013 

LRIP  6 

$1,135,  527,863 

36/2 

FY  2014 

LRIP  7 

TBD 

35* 

FY  2015 

LRIP  8 

TBD 

* 

no 

■vt 

FY  2016 

LRIP  9 

TBD 

57* 

FY  2017 

LRIP  10 

TBD 

96* 

FY  2018 

LRIP  11 

TBD 

121* 

FY  2019 

FRP  1 

TBD 

160* 

Starting  in  FY  2020 

Projected  numbers  for  future  orders 


1  LRIP  as  defined  by  the  Defense  Acquisition  University  is  the  "effort  intended  to  result  in  completion  of  manufacturing 
development  in  order  to  ensure  adequate  and  efficient  manufacturing  capability  and  to  produce  the  minimum  quantity 
necessary  to  provide  production  or  production-representative  articles  for  Initial  Operational  Test  and  Evaluation." 
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(FOUO)  Quality  Management 
System  Inspection 

We  inspected  Pratt  &  Whitney’s  quality  management  system  for  the  F135  program 
at  their  East  Hartford  and  Middletown,  Connecticut,  facilities.  We  evaluated  the 
quality  management  system  policies,  processes,  and  procedures  for  compliance 
to  the  AS9100C,  "Quality  Management  System,"  standard.  We  also  evaluated 
the  F-35  JPO  and  the  Defense  Contract  Management  Agency’s  (DCMA’s)  program 
management  and  oversight  of  Pratt  &  Whitney’s  quality  management  system  for 
the  F135  program. 

We  identified  a  total  of  61  nonconformities,  classified  as  major  or  minor,  that  were 
considered  violations  of  AS9100C,  regulatory  requirements,  and  DoD  policies.  As 
defined  by  the  AS9101,  "Quality  Management  Systems  Audit  Requirements  for 
Aviation,  Space,  and  Defense  Organizations,"  standard,  a  major  nonconformity 
is  a  nonfulfillment  of  a  requirement  that  is  likely  to  result  in  the  failure  of  the 
quality  management  system  or  reduce  its  ability  to  ensure  controlled  processes 
or  compliant  products/services.  A  minor  nonconformity  is  a  nonfulfillment  of  a 
requirement  that  is  not  likely  to  result  in  the  failure  of  the  quality  management 
system  or  reduce  its  ability  to  ensure  controlled  processes  or  compliant  products 
or  services. 

Table  2  shows  the  breakdown  of  major  and  minor  nonconformities,  and 
opportunities  for  improvement  (OFIs)2  found  against  Pratt  &  Whitney,  the  F-35  JPO, 
and  DCMA.  Figure  3  shows  the  nonconformities  directed  to  Pratt  &  Whitney  by 
AS9100C  clause.  The  following  sections  discuss  major  nonconformities  only  and 
are  organized  by  its  applicable  criteria. 


Table  2.  Major/Minor  Nonconformities  and  OFIs  for  Each  Organization 


Organization 

Major 

Minor 

Opportunity  for 
Improvement 

Pratt  &  Whitney 

33 

19 

1 

DCMA 

3 

0 

0 

F-35  JPO 

5 

1 

1 

2  An  OFI  is  an  industry  best  practice  when  a  specific  requirement  does  not  exist. 
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Figure  3.  Pratt  &  Whitney  Nonconformities  by  AS9100  Clause 


(FOUO)  Results  of  Pratt  &  Whitney  AS9100C 
Conformance  Inspection 

Documentation  Requirements  (4.2) 

Pratt  &  Whitney  did  not  establish  a  specific  quality  program  plan  that  described 
the  management  of  unique  F135  program  quality  assurance  requirements.  The 
SDD  contract  required  that  Pratt  &  Whitney  maintain  a  quality  management 
system  and  that  a  quality  plan  be  summarized  in  the  body  of  the  Propulsion 
System  Development  Plan  (PSDP).  In  the  PSDP,  Pratt  &  Whitney  formally 
requested  relief  to  eliminate  the  requirement  to  establish  a  quality  plan  for  the 
F135  program;  however,  the  F-35  JPO  did  not  formally  relieve  Pratt  &  Whitney 
of  this  requirement.  The  F-35  JPO  and  Pratt  &  Whitney  agreed  to  follow 
Pratt  &  Whitney’s  corporate-level  quality  manual;  however,  this  quality  manual 
described  corporate  policy  on  quality  assurance  and  not  unique  F135  engine’s 
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quality  assurance  requirements.  The  SDD  contract  statement  of  work  (SOW) 
identified  unique  program  quality  assurance  requirements  that  were  not  addressed 
by  Pratt  &  Whitney’s  quality  manual  such  as  product  specialty  engineering 
discipline  elements,  critical  manufacturing  and  supporting  processes,  and  technical 
performance  measures.  Due  to  the  multi-mission  capability  requirements  of  the 
F-35  and  criticality  of  a  single  engine  design,  an  F-35  JPO-approved  F135  program 
quality  plan  is  necessary  to  capture  unique  quality  assurance  goals  and 
requirements  of  the  program. 

(FGUQ)  Management  Commitment  (5.1) 

(FOUO)  Pratt  &  Whitney  management  failed  to  ensure  that  process  capability  (Cpk)3 
goals  are  being  met,  nonconformances  per  engine  and  quality  escapes  are 
being  reduced,  and 

mmm  Pratt  &  Whitney  identified  that  41  percent  of  all  key  product 
characteristics  (KPC)4  do  not  meet  their  process  certification  requirements, 
which  include  a  Cpk  of  1.33  or  better.  A  process  with  a  Cpk  of  1.33  or  better  is 
a  capable  and  stable  process.  Moreover,  Pratt  &  Whitney  has  not  developed 
process  improvement  plans  for  60  percent  of  those  underperforming  KPCs.  Given 
the  significant  percentage  of  underperforming  KPCs,  the  F-35  JPO  should  expect 
escalating  rates  of  quality  escapes  and  nonconformances  per  engine,  which  may 
impact  engine  performance,  schedule,  and  cost. 


(FOUO) 


If  Pratt  &  Whitney  does  not  make  continuous 
improvement  top  F135  program  priorities,  then  product  quality  will  continue  to 
decline  especially  with  increasing  production  quantities  in  the  future. 


3  Cpk:  Capability  index  that  defines  the  relationships  of  the  "centeredness"  (or  mean)  and  "spread"  (or  standard  deviation) 
of  capability  to  specification  limits.  Cpk  measures  the  accuracy  of  the  process. 

4  KPC:  Key  Product  Characteristics  define  the  unique  requirements  for  associated  manufacturing  inputs  and  inspection 
sampling  rates.  KPC  are  applied  to  certain  drawing  features  and  require  process  control  to  evaluate  inputs,  outputs, 
data  collection  and  control  plans  to  monitor  the  manufacturing  inputs.  A  process  is  considered  "Certified"  when  the 
process  output  is  in  control,  capable,  and  has  a  Cpk  of  1.33  or  better. 
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Planning  (5.4) 

Pratt  &  Whitney  is  contractually  required  to  establish  program  objectives  and 
present  quality  assurance  metrics  in  an  open  forum  with  the  F-35  JPO  during 
the  monthly  program  reviews  (MPRs).  Pratt  &  Whitney  has  not  established 
program  objectives  that  are  mutually  agreed  upon  between  Pratt  &  Whitney 
and  F-35  JPO  for  the  SDD  contract.  There  were  program  objectives  in  2011; 
however,  those  program  objectives  were  no  longer  relevant  and  no  new  objectives 
had  been  defined.  In  addition,  LRIP  Contract  SOWs  required  Pratt  &  Whitney 
to  report  quality  assurance  metrics  such  as  process  variability  reduction  and 
nonconformance  parts  per  million  at  MPRs.  Metrics  were  instead  provided 
electronically  at  the  integrated  product  team  level  and  may  not  have  received 
management  and  interdisciplinary  team  visibility  to  ensure  that  quality 
assurance  metrics  were  improving.  Not  reporting  program  objectives  and 
quality  metrics  at  MPRs  or  at  required  intervals  limits  the  ability  of  the  F-35  JPO 
to  make  programmatic  decisions  based  upon  the  latest  available  data.  As  of 
November  2014,  the  F-35  Program  Executive  Officer  redirected  the  program  to 
establish  F135  specific  goals  in  future  production  contracts. 

Planning  of  Production  Realization  (7.1) 

Pratt  &  Whitney  personnel  did  not  always  follow  personal  protective 
equipment  (PPE)  requirements,  putting  themselves  at  significant  risk  for  injury. 

For  example,  an  acid  dipping  process,  also  called  "pickling,"  required  that  a  face 
mask  be  worn  in  the  vicinity  of  the  acid  tanks.  In  one  instance,  an  operator  was 
not  wearing  a  face  mask  while  draining  a  water  tank  that  was  near  the  acid  tanks; 
however,  it  was  unclear  whether  a  face  mask  was  required  due  to  ambiguous 
designations  for  the  limits  of  areas  requiring  PPE.  Other  examples  include 
personnel  not  consistently  wearing  protective  gloves  at  an  electrical  discharge 
machine  station  and  personnel  not  wearing  lab  coats  in  the  blade  vapor  coating 
area.  PPE  practices  must  be  followed  to  reduce  risk  of  injury  to  personnel  or 
contamination  of  product. 

Project  Management  (7.1.1) 

Pratt  &  Whitney  did  not  track  and  manage  delinquent  contract  data  requirements 
list  (CDRL)5  items  to  ensure  deliveries  occur  in  accordance  with  contractual 
requirements.  In  addition,  Pratt  &  Whitney  contracts  management  did  not 
have  a  closed-loop  corrective  action  system  for  delinquent  or  rejected  CDRLs. 

Pratt  &  Whitney  had  approximately  390  delinquent  CDRLs  dating  to  2000  that  were 


5  According  to  Defense  Acquisition  University,  the  CDRL  is  a  "list  of  authorized  data  requirements  for  a  specific 
procurement  that  forms  part  of  the  contract. ..The  CDRL  is  the  standard  format  for  identifying  potential  data 
requirements  in  a  solicitation,  and  deliverable  data  requirements  in  a  contract." 
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never  delivered  to  F-35  JPO.  If  Pratt  &  Whitney  does  not  ensure  CDRLs  are  tracked, 
submitted,  and  managed,  then  the  F-35  JPO  may  not  receive  critical,  accurate,  or 
relevant  program  information  necessary  to  verify  the  design,  implementation,  and 
product  performance. 

Since  2011  and  throughout  the  SDD,  LRIP  5,  6,  and  7  contracts,  the  F-35  JPO 
rejected  the  Integrated  Master  Schedules  (IMSsJ/Integrated  Master  Plans  (IMPs) 
due  to  inaccurate  and  incomplete  information.  The  IMS/IMP  did  not  capture  major 
supplier  data,  which  was  critical  to  capture  the  total  program  and  meet  basic 
scheduling  guidelines.  Additionally,  there  were  several  open  DCMA  corrective 
action  requests  (CARs)  written  against  Pratt  &  Whitney  and  Rolls  Royce,  noting 
numerous  concerns  with  the  IMS/IMP,  including  poor  quality  of  subcontractor 
schedule  validation  and  analysis.  The  failure  of  Pratt  &  Whitney  to  provide  the 
F-35  JPO  with  an  acceptable  IMS/IMP  may  adversely  affect  its  ability  to  address 
schedule  risks. 

Risk  Management  (7.1.2) 

The  risk  management  program  at  Pratt  &  Whitney  did  not  provide  management 
visibility  and  awareness  of  risks  that  can  adversely  affect  the  program. 

Pratt  &  Whitney’s  risk  management  plan  stated  that  F135  Risk  and  Issues 
Management  Board  (RIMB)  will  review  each  integrated  product  team’s  risk 
and  determine  which  should  be  elevated  to  a  program  or  system  level  risk. 

However,  we  found  that  the  RIMB  only  discussed  technical  risks  and  not 
production,  sustainment,  design,  or  supply  chain  risks.  The  risk  management 
database  identified  162  active  risks;  however,  the  F135  RIMB  presentation  dated 
October  21,  2014  only  discussed  10  active  risks.  Additionally,  we  found  insufficient 
evidence  that  the  required  technical  leads  were  present  during  RIMB  to  make 
decisions  on  ranking  risk,  risk  mitigation  plans,  and  removing  risk. 

Pratt  &  Whitney  generated  software  issues  at  the  software  development  integrated 
product  team  level;  however,  we  found  no  evidence  that  these  issues  were  analyzed 
to  determine  system  level  risks.  For  example,  there  were  at  least  40  open  safety 
critical  technical  issues  at  the  integrated  product  team  level  without  evidence  of 
risk  analysis  to  determine  if  the  issues  were  potential  software  risks  that  should  be 
elevated  to  the  RIMB. 

Pratt  &  Whitney’s  risk  management  database  was  not  maintained  and  did  not 
reflect  accurate  risk  information  including  risk  category,  status,  mitigation 
plans,  and  risk  ranking.  The  risk  management  procedure  defined  a  structure  for 
identifying  and  categorizing  risks;  however,  the  risk  management  database  did 
not  have  the  same  risk  categories.  We  found  that  several  risks  were  duplicated 
in  multiple  folders  of  the  database  where  the  same  risk  was  open  in  one  category 
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but  already  closed  in  another.  Lastly,  risk  mitigation  plans  were  not  developed  for 
production,  sustainment,  and  design  risks.  If  the  F-35  JPO  does  not  ensure  that 
Pratt  &  Whitney  maintain  an  effective  risk  management  program  with  rigor  and 
discipline  to  identify,  evaluate,  and  mitigate  risks,  then  significant  risks  and  issues 
that  should  be  managed  and  tracked  at  the  system  level  may  be  overlooked. 

Configuration  Management  (7.1.3) 

The  Pratt  &  Whitney  Software  Quality  Assurance  (SQA)  organization  did  not 
perform  physical  configuration  audits  (PCAs)  or  functional  configuration 
audits  (FCAs)  on  any  computer  software  configuration  items  (CSCIs)  in  accordance 
with  internal  procedures.  A  review  of  SQA  records  for  the  main  engine,  lift  system, 
and  boot  CSCIs  showed  no  evidence  that  SQA  performed  PCAs  and  FCA  activities. 
Additionally,  future  SQA  audit  schedules  did  not  contain  PCAs  and  FCAs.  A  PCA  is 
a  formal  examination  of  the  as-built  configuration  of  a  CSCI  against  its  technical 
documentation,  to  establish  the  product  baseline  (as-built  vs.  as-designed). 

An  FCA  verifies  that  the  CSCI  meets  the  requirements  documented  in  the  Software 
Requirements  Specification  and  the  Interface  Requirements  Specification. 

Not  conducting  PCAs  and  FCAs  on  all  CSCIs  can  result  in  nonconforming  software 
being  integrated  into  the  F-35  aircraft  with  functionalities  that  are  highly 
dependent  on  software. 

Design  and  Development  (7.3) 

Design  and  Development  Planning  (7.3.1) 

The  F135  Software  Development  Plan  (SDP)  did  not  describe  the  current  software 
architecture,  did  not  identify  all  software  configuration  items,  and  did  not  define 
all  applicable  product  software  levels  (PSLs).6  The  SDP  described  an  outdated 
software  architecture  where  the  prognostics  and  health  management  (PHM) 
software  was  a  separate  CSCI.  In  the  current  software  architecture,  PHM  software 
is  embedded  into  the  main  engine  and  lift  system  CSCIs.  The  SDP  also  lacked  a 
description  of  the  boot  CSCI.7  Lastly,  the  SDP  did  not  define  PSL  requirements  for 
software  designated  lower  than  PSL1  or  non-safety  critical  software. 

When  the  PHM  software  was  designed  as  its  own  CSCI,  it  was  designated  as  a  PSL3. 
Because  it  is  now  embedded  in  the  main  engine  and  lift  system  CSCIs  that  are  both 
designated  as  PSL1,  the  PHM  must  be  developed  and  tested  to  PSL1  requirements, 
unless  Pratt  &  Whitney  can  demonstrate  that  their  design,  development,  and 


6  Product  software  levels  (PSLs)  are  safety  critical  categories  for  software  products.  PSL1,  the  highest  safety  critical 
category,  are  designated  for  software  whose  anomalous  behavior  would  cause  or  contribute  to  a  failure  of  a  system 
function  and  result  in  a  catastrophic  failure. 

7  The  boot  is  the  software  system  that  provides  a  starting  point  in  the  event  that  a  Full  Authority  Digital  Engine  Control 
or  Propulsion  Health  Area  Manager  is  started  or  reset. 
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test  processes  ensure  that  lower  level  PSL-partitioned  software  won’t  adversely 
affect  the  main  engine  and  lift  system  software.  Flight  application  software  that 
is  not  developed  and  tested  in  accordance  with  PSL  requirements  could  impact 
performance  of  critical  software  and  could  introduce  a  catastrophic  system  failure. 

Design  and  Development  Outputs  (7.3.3) 

Software  safety  requirements  were  not  completely  implemented  for  the  boot 
CSCI  as  evidenced  by  34  safety  requirements  that  were  not  traceable  to  software 
design  and  test  documents.  For  example,  there  was  no  evidence  that  the  safety 
requirement  for  the  boot  software  to  provide  fail-safe  recovery  from  inadvertent 
instruction  jumps  was  met.  There  was  also  no  evidence  that  the  safety 
requirement  for  operational  program  loads  to  not  contain  unused  executable  code 
was  met.  Incomplete  requirements  verification  for  both  the  main  engine  and  lift 
fan  CSCIs  could  create  a  performance  and  safety  risk  for  the  F135  program. 

Pratt  &  Whitney  did  not  identify  critical  characteristics  for  many  of  the  critical 
safety  items  (CSIs)  on  the  SOW  CSI  list.  Also,  the  CSI  SOW  required  that 
Pratt  &  Whitney  ensure  all  drawings  and  associated  technical  data  clearly  identify 
that  the  item  is  a  CSI;  however,  there  were  no  CSI  markings  on  any  drawing  or 
technical  data.  The  purpose  of  identifying  CSIs  on  drawings  and  associated 
technical  data  is  to  inform  stakeholders  that  the  parts  being  manufactured, 
dispositioned,  tested,  or  maintained  are  critical  to  the  safety  of  the  aircraft  and 
personnel.  The  failure  to  identify  CSIs  decreases  the  awareness  and  ability  of 
F135  personnel  responsible  for  design,  development,  production,  test,  maintenance, 
and  inspection  to  ensure  that  additional  CSI  controls  are  implemented  for  parts 
that  are  critical  to  the  safety  of  the  aircraft  and  personnel. 

Design  and  Development  Verification  (7.3.5) 

The  Pratt  &  Whitney  SQA  organization  did  not  conduct  Software  Life  Cycle  (SLC) 
audits  on  critical  software  suppliers  since  2009,  even  though  there  have  been 
recent  software  releases.  Pratt  &  Whitney  SQA  procedures  required  SLC  audits 
to  be  conducted  on  suppliers  for  all  software  released.  SLC  audits  include,  but 
are  not  limited  to,  project  planning  and  tracking,  requirements,  design,  code,  test 
plans  and  procedures,  test  reports,  software  documentation,  software  development 
tools,  configuration  management,  library  controls,  shipping,  handling,  and 
delivery  processes. 

The  Pratt  &  Whitney  SQA  organization  could  not  provide  sufficient  evidence 
that  they  performed  software  audits  in  accordance  with  their  software  quality 
assurance  procedure.  Examples  of  software  audit  activities  include  reviewing 
change  requests,  witnessing  code  and  unit  testing,  and  reviewing  software  release 
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procedures  and  packages.  If  audits  are  not  conducted  on  Pratt  &  Whitney-  and 
supplier-developed  software,  then  there  is  no  assurance  that  quality  assurance 
practices  are  being  implemented  for  critical  software  products. 

Design  and  Development  Verification  and  Validation  Testing  (7.3. 6.1) 

The  F135  Electronics  Verification  Bench  (EVB)  laboratory  was  not  formally 
configuration  controlled.  Inventory  of  EVB  equipment  racks,  software  change 
requests,  and  problem  reports  were  informally  managed  by  e-mails  and  an  excel 
spreadsheet.  In  addition,  the  configuration  of  the  laboratory  can  be  altered  or 
interrupted  during  formal  test  by  other  remote  users.  The  EVB  simulates  the 
entire  propulsion  system  while  using  electronics  that  are  functionally  equivalent 
to  flight  hardware.  The  EVB  laboratory  was  used  for  software  configuration 
item  tests,  integration  tests,  failure  mode  and  effects  tests,  acceptance  tests,  and 
system  evaluation  tests.  The  SDP  required  that  software  test  verification  uses  a 
configuration  controlled  set  of  test  assets  with  test  log  files,  software  versions, 
and  configuration  of  the  EVB.  Pratt  &  Whitney’s  lack  of  formal  configuration 
management  control  in  the  EVB  laboratory  may  invalidate  verification  test  results 
on  critical  software  verification  test  activities. 

Control  of  Design  and  Development  Changes  (7.3.7) 

Pratt  &  Whitney  could  not  provide  evidence  of  required  engineering  technical 
reviews  and  approvals  for  specification  change  requests  and  component 
requirement  change  requests.  There  was  no  evidence  of  integrated  product  team 
review  and  approvals  in  accordance  with  the  F135  specification  change  request 
procedure  for  approximately  100  records  reviewed.  There  was  also  no  evidence  of 
management  approvals  for  component  requirement  change  requests  of  Rolls-Royce 
components.  The  lack  of  technical  review  can  lead  to  specification  changes  being 
implemented  without  adequate  analysis  on  impacts  to  the  product  or  system. 

Purchasing  (7.4) 

Purchasing  Process  (7.4.1) 

Pratt  &  Whitney’s  minimum  baseline  criteria  for  selecting  suppliers,  such 
as  performance/quality  history  and  cost  and  schedule  data,  were  not  clearly 
documented  within  its  procurement  policies  and  procedures.  Pratt  &  Whitney 
also  did  not  clearly  document  the  actions  to  take  when  a  supplier  continues  to 
have  a  poor  quality  rating  or  is  considered  a  high  risk.  Several  suppliers  had 
been  rated  high  risk  for  extended  periods  of  time.  We  found  that  one  supplier 
had  been  considered  high  risk  for  33  consecutive  months.  If  criteria  for  selection 
and  management  of  suppliers  are  not  established,  clearly  documented,  and 
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implemented,  Pratt  &  Whitney  cannot  provide  assurance  that  their  suppliers  are 
qualified  to  deliver  quality  products  that  meet  all  quality,  schedule,  cost,  and 
performance  requirements. 

Production  and  Service  Provision  (7.5) 

Validation  of  Processes  for  Production  and  Service  Provision  (7.5.2) 

Pratt  &  Whitney  did  not  recertify  Level  3  nondestructive  test  (NDT)8  personnel 
within  a  five  year  period  in  accordance  with  international  standards,  such  as  the 
National  Aerospace  Standard  410,  "Certification  &  Qualification  of  Nondestructive 
Test  Personnel."  Several  Level  3  personnel  were  initially  certified  in  the  early 
1990s;  however,  they  were  not  recertified  within  a  five  year  period.  We  found  that 
Level  3  personnel  were  initially  certified  to  NDT  techniques  such  as  Etch  Inspection 
Method  1979,  Florescent  Penetration  Inspection  1980,  Thickness  Determination 
(Ultrasonic)  Inspection  1981,  Radiographic  Inspection  1983,  and  Eddy  Current 
Testing  1999.  If  Pratt  &  Whitney  does  not  ensure  that  Level  3  personnel  are 
periodically  recertified  to  nationally  recognized  criteria,  then  the  skills  necessary 
to  determine  product  acceptance  using  NDT  methods  may  be  degraded. 

Customer  Property  (7.5.4) 

Pratt  &  Whitney’s  handling  of  electrostatic  discharge  sensitive  (ESDS)  Government 
property  did  not  meet  EVB  laboratory  requirements  as  evidenced  by  the  lack  of 
the  following:  equipment  ground  straps,  wrist  straps,  verification  of  resistivity  in 
ground  and  wrist  straps,  protective  coverings  for  ESDS  equipment,  and  grounding 
of  fixtures  that  hold  critical  ESDS  components.  The  EVB  laboratory  used  non-flight 
full  authority  digital  engine  control  (FADEC)  Government  property  to  perform 
qualification  testing  of  flight  application  software.  If  ESDS  protective  procedures 
are  not  followed,  there  is  a  risk  of  latent  defects  and  damage  to  Government 
property  and  qualification  test  equipment. 

Preservation  of  Product  (7.5.5) 

The  F135  engine  final  assembly  area  was  neither  adequately  identified  as  foreign 
object  damage  (FOD)-controlled  area  nor  protected  from  common  FOD  items.  The 
F135  engine  final  assembly  area  had  FOD  signage  at  the  two  entry  lobbies  of  the 
building;  however,  there  were  no  FOD  signage  within  the  engine  final  assembly 
areas.  Pratt  &  Whitney  FOD  procedures  also  did  not  prevent  common  FOD  items 
from  entering  the  engine  final  assembly  area.  For  example,  items  such  as  safety 


8  Level  3  NDT  is  the  highest  level  of  NDT  certification.  Level  NDT  personnel  are  certified  in  accordance  with 

National  Aerospace  Standard  410  in  one  or  more  NDT  methods  and  shall  have  a  thorough  knowledge  of  the  written 
instructions,  codes,  specifications  and  standards  used  by  the  employer.  He/she  shall  also  have  a  thorough  knowledge  of 
the  materials,  components,  product  technologies,  NDT  methods,  and  NDT  techniques  used  by  the  employer. 
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and  tie  wire,  damaged  storage  brackets,  empty  consumable  cans  (lubricants, 
cleaners),  and  packaging  materials  (styrofoam,  plastic  wrap,  cardboard)  were 
found  in  the  final  assembly  area.  If  common  FOD  items  are  not  restricted  from 
FOD  controlled  areas,  then  there  is  an  increased  risk  of  hardware  contamination 
and  damage  on  assembled  and  delivered  engines. 

Control  of  Monitoring  and  Measuring  Equipment  (7.6) 

Pratt  &  Whitney  did  not  ensure  that  calibration  equipment  records  were  complete 
and  obsolete  equipment  calibration  labels  were  removed.  Certificates  of  calibration 
for  coordinate  measurement  machines  (CMMs)  were  missing  critical  calibration 
data  such  as  temperature  and  relative  humidity  conditions,  measurement 
uncertainties,  and  temperature  standard.  Additionally,  CMM  limited  calibration 
labels  did  not  reflect  actual  calibration  status  of  the  equipment.  In  one  instance, 
previous  limited  calibration  labels  with  obsolete  calibration  data,  such  as 
compensation  factors,  were  still  attached  to  the  CMM.  CMMs  were  used  to  accept 
and  inspect  engine  components  in  various  stages  of  assembly  and  manufacturing. 
The  use  of  incomplete  or  outdated  calibration  measurement  data  can  affect  product 
measurements  that  are  used  for  product  acceptance  and  increase  product  rework. 

Measurement ,  Analysis  and  Improvement  -  General  (8.1) 

Pratt  &  Whitney  did  not  consistently  analyze  and  report  KPC  Cpk  data  for 
F135  engine  hardware.  We  found  several  inconsistencies  among  Cpk  values 
reported  from  multiple  sources  for  the  same  KPCs.  Furthermore,  Pratt  &  Whitney 
manufacturing  engineers  could  not  identify  what  analysis  or  source  that  was 
used  to  support  Cpk  results  in  the  database  report.  DCMA  used  the  Cpk  database 
report  to  establish  a  mandatory  Government  inspection  point  for  all  KPCs  that 
have  a  Cpk  of  less  than  1.  The  F-35  JPO  also  monitored  the  data  as  an  indication  of 
technical  and  program  risk.  If  KPC  data  are  not  correctly  analyzed  and  reported, 
then  Pratt  &  Whitney,  F-35  JPO,  and  DCMA  cannot  make  sound  program  and 
resource  management  decisions,  nor  determine  if  engine  hardware  processes 
are  stable. 

Control  of  Nonconforming  Product  (8.3) 

Pratt  &  Whitney  did  not  always  identify  CSI  parts,  their  impacted  critical  and 
major  characteristics,  and  the  correct  disposition  status  of  affected  CSIs  on  major 
variance  requests  (MVRs)  also  known  as  requests  for  waivers  and  deviations. 

For  example,  the  augmentor  fuel  pump,  contractually  identified  as  a  CSI,  was 
not  identified  as  a  CSI  on  an  MVR.  In  addition,  Pratt  &  Whitney  combined  the 
definitions  of  critical  and  major  variances  under  one  category  called  major 
variances,  which  was  in  violation  of  contractual  requirements.  Variance  requests 
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that  do  not  clearly  identify  items  as  CSIs  may  not  be  reviewed  and  dispositioned 
in  accordance  with  CSI  program  requirements.  Furthermore,  the  misclassification 
and  unclear  identification  of  a  CSI  or  its  critical  characteristic  can  lead  to  errors  in 
processing  and  dispositioning  of  associated  variance  requests. 

Improvement  (8.5) 

Corrective  Action  (8.5.2) 

Pratt  &  Whitney  did  not  always  document  root  cause  and  corrective  actions  for 
nonconformances.  For  example,  root  cause  or  corrective  action  details  were  not 
documented  on  three  1st  Stage  Axial  Compressor  Rotor  Assembly  nonconformances, 
which  was  in  violation  of  internal  procedures.  In  addition,  Pratt  &  Whitney  did 
not  provide  timely  review  and  verification  of  CARs  they  issued  to  suppliers.  In 
many  cases,  the  supplier  responded  with  a  root  cause  corrective  action  plan  by 
the  due  date;  however,  Pratt  &  Whitney  did  not  review  and  close  out  the  CARs  in 
a  reasonable  timeframe,  including  some  CARs  that  were  not  closed  until  almost 
two  years  later.  If  the  root  cause  and  corrective  actions  are  not  documented 
and  reviewed  for  adequacy  in  a  timely  manner,  then  there  is  a  risk  of  continued 
reoccurrence  of  nonconformances. 

Government  Quality  Assurance  Oversight 

Performance  of  the  F-35  Joint  Program  Office 

We  found  nonconformities  with  the  F-35  JPO’s  quality  assurance  oversight  of 
the  F135  program.  Major  nonconformities  include:  insufficient  CSI  program 
requirements,  inadequate  Government  contract  quality  assurance  (GCQA) 
requirements  flow  down  for  CSIs,  misapplication  of  the  MVR  process,  and  no 
requirement  for  approval  of  CDRL  items.  Additional  quality  assurance  oversight  is 
required  to  ensure  program  cost,  schedule,  and  performance  goals  are  attained. 

Critical  Safety  Item  Management 

Public  Law  108-136,  Section  802,  "Quality  control  in  procurement  of  aviation 
CSIs  and  related  services,”  requires  DoD  to  prescribe  a  quality  control  policy 
for  the  procurement  of  aviation  CSIs.9  The  Joint  Service  CSI  Instruction, 
SECNAVINST  4140.2/AFI20-106,  "Management  of  Aviation  Critical  Safety  Items," 
dated  January  25,  2006  implements  Public  Law  108-136  and  is  applicable  to 


9  Title  10,  Section  2319  of  the  United  Sates  Code  is  the  applicable  statute  that  implements  aviation  CSI  requirements  of 
Public  Law  108-136,  Section  802. 
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Program  Executive  Officers,  commanders  of  system  acquisition  and  logistics 
organizations,  program  managers,  and  other  agencies  that  procure,  repair,  or 
service  aviation  materiel.  The  Joint  Service  CSI  Instruction  defines  CSI  as: 

A  part,  assembly,  installation  equipment,  launch  equipment, 
recovery  equipment,  or  support  equipment  for  an  aircraft  or 
aviation  weapons  system  that  contains  a  characteristic  any  failure, 
malfunction,  or  absence  of  which  could  cause  a  catastrophic  or 
critical  failure  resulting  in  the  loss  or  serious  damage  to  the 
aircraft  or  weapons  system,  an  unacceptable  risk  of  personal 
injury  or  loss  of  life,  or  an  uncommanded  engine  shutdown  that 
jeopardizes  safety. 

CSIs  require  special  handling,  engineering,  manufacturing,  and  inspection 
documentation  to  control  and  ensure  safety  of  flight  and  prevent  potential 
catastrophic  or  critical  consequences  of  failure.  Supplemental  guidance  for  the 
Joint  Service  CSI  Instruction  is  provided  in  the  Joint  Aeronautical  Commanders 
Group  (JACG)  Aviation  Source  Approval  and  Management  Handbook. 

The  F-35  JPO  required  Pratt  &  Whitney  to  implement  a  CSI  management 
program  according  to  the  CSI  SOW.  When  the  CSI  SOW  was  initially  developed, 
the  Joint  CSI  Instruction  did  not  exist  to  implement  Public  Law  108-136.  The 
CSI  SOW  implemented  a  tailored  version  of  NAVAIR  Instruction  4200. 25D, 
"Management  of  Critical  Application  Items  Including  Critical  Safety  Items,"  dated 
June  20,  2002.  The  CSI  SOW  was  not  modified  to  implement  the  full  scope  of  the 
Joint  Service  CSI  Instruction  for  contracts  awarded  after  the  instruction’s  release 
date  (LRIPs  2  through  7).  Nevertheless,  we  found  that  the  CSI  program  did  not 
meet  NAVAIR  Instruction  4200. 25D  or  Joint  Service  CSI  Instruction  in  several  areas. 
For  example, 

•  CSI  parts  were  identified  based  on  the  part’s  probability  of  failure  rather 
than  the  consequence  of  failure. 

•  The  top  level  F135  engine  assembly  is  not  identified  as  a  CSI  on  the 

CSI  SOW  even  though  it  contains  at  least  one  critical  safety  characteristic. 

•  CSIs  are  identified  on  the  CSI  SOW  by  nomenclature  (a  family  of  parts) 
and  not  by  individual  part  number.  Therefore,  the  list  does  not 
specifically  identify  which  parts  and  assemblies  are  CSI. 

•  Not  all  CSIs  on  the  CSI  SOW  identify  associated  critical 
characteristics.  Some  identify  major  characteristics  and  some  have 
undefined  characteristics. 

•  There  was  no  document  that  identified  prime  contractors,  original 
equipment  manufacturers,  and  alternate  sources  for  CSIs. 
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•  There  was  no  evidence  that  a  joint  CSI  program  assessment  has  been  done 
annually  or  periodically  per  the  CSI  SOW. 

•  The  F-35  JPO  did  not  adequately  flow  down  CSI  GCQA  oversight 
requirements  for  part  procurements  to  DCMA.  Per  the  F-35  JPO/DCMA 
Memorandum  of  Agreement  (MOA),  GCQA  is  only  required  for  parts 
containing  critical  characteristics. 

The  F135  CSI  program’s  lack  of  compliance  to  the  NAVAIR  Instruction  4200. 25D 
or  Joint  Service  Instruction  increases  the  risk  of  nonconforming  CSI  parts 
being  installed  on  aircraft.  It  also  increases  the  risk  that  parts  critical  to  the 
safety  will  not  be  properly  handled  and  controlled  by  production,  test,  and 
maintenance  personnel. 

Federal  Acquisition  Regulation  46.407,  “ Nonconforming  Supplies  or  Services" 
The  F-35  JPO  used  the  variance  process  to  waive  requirements  for  extended 
periods  of  time,  which  was  a  misapplication  of  the  variance  request  process  and 
did  not  discourage  the  repeated  tender  of  nonconforming  supplies  or  services  in 
accordance  with  Federal  Acquisition  Regulation  (FAR)  46.407,  "Nonconforming 
Supplies  or  Services."  As  defined  by  Military  Standards  MIL-STD-973,  "Military 
Standard  Configuration  Management,"  a  deviation  (or  a  variance)  is, 

A  specific  written  authorization,  granted  prior  to  the  manufacture 
of  an  item,  to  depart  from  a  particular  requirement(s)  of  an  item's 
current  approved  configuration  documentation  for  a  specific 
number  of  units  or  a  specified  period  of  time.  (A  deviation  differs 
from  an  engineering  change  in  that  an  approved  engineering  change 
requires  corresponding  revision  of  the  item’s  current  approved 
configuration  documentation,  whereas  a  deviation  does  not.) 

The  F-35  JPO  performed  a  system  level  F135  FCA  that  identified  175  major 
variances  to  specification  requirements;  however,  the  F-35  JPO  proceeded  to 
establish  the  Initial  Service  Release  (ISR)  baseline  in  2010  with  these  deficiencies. 
Currently,  twenty  percent  of  the  original  175  MVRs  remain  open.  Some  MVRs 
date  back  to  2009.  The  five  longest  open  MVRs  have  closure  plans  that  have  been 
extended  multiple  times  for  periods  of  1  to  7  years.  Additionally,  the  F-35  JPO 
and  Pratt  &  Whitney  estimate  that  two  percent  of  those  MVRs  will  be  closed  by 
specification  changes. 

According  to  the  F-35  JPO,  reoccurring  MVRs  are  due  to  the  lot-to-lot  program 
concurrency,  which  has  required  engine  production  to  proceed  in  order  to  meet 
the  aircraft  production  schedule.  Their  plan  is  to  close  all  MVRs  by  the  end  of 
SDD  in  2016.  The  F-35  JPO  will  continue  to  accept  nonconforming  engines  if 
MVR  extensions  continue  to  be  approved  and  not  resolved. 
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DoD  Directive  5000.01,  "The  Defense  Acquisition  System ” 

The  F-35  JPO  awarded  SDD  and  LRIP  contracts  with  CDRL  items  that  were 
delivered  for  information  only,  and  not  for  approval.  Many  of  the  program  planning 
and  schedule  CDRL  items  were  removed  from  current  sustainment  and  production 
contracts.  Examples  of  plans  that  were  submitted  for  information  only  included: 
Configuration  Management  Plan,  IMS/IMP,  Quality  Plan,  Risk  Management  Plan, 
Software  Development  Plan,  System  Engineering  Management  Plan,  Engineering 
Change  Proposals,  and  Variance  Requests.  According  to  DoD  Directive  5000.01, 
"The  Defense  Acquisition  System,”  the  program  manager  has  the  responsibility 
for  and  authority  to  accomplish  program  objectives  for  development,  production 
and  sustainment  and  shall  be  accountable  for  credible  cost,  schedule,  and 
performance.  The  absence  of  F-35  JPO  approval  on  program  planning  and  schedule 
CDRL  items  limits  the  ability  of  the  government  to  provide  F135  program  direction 
and  oversight. 

Performance  of  DCM A 

We  documented  DCMA  nonconformities  in  three  areas:  CDRL  reviews,  corrective 
action  request  management  and  recordkeeping,  and  variance  request  evaluation 
and  approval.  If  DCMA  does  not  provide  the  required  contract  administration  and 
quality  assurance  oversight,  then  there  is  additional  risk  that  delivered  engines 
may  not  meet  reliability  requirements. 

DCMA  Instruction  205,  “ Major  Program  Support" 

DCMA  did  not  adequately  review  CDRL  items  that  could  impact  program 
surveillance  activities.  DCMA  was  not  included  in  the  distribution  lists  for 
several  CDRL  items  on  the  LRIP  6  and  7  contracts  such  as  the  cost  data  summary 
report,  contract  work  breakdown  structure,  engineering  change  proposal,  and 
variance  request.  According  to  DCMA  Instruction  205,  "Major  Program  Support”, 
DCMA  shall  review  the  CDRLs  to  identify  any  deliverables  that  impact  DCMA 
program  surveillance  to  ensure  DCMA  is  included  in  the  distribution.  DCMA 
Instruction  205  also  requires  that  DCMA  contact  the  F-35  JPO  contracting  officer 
to  resolve  any  conflicts  between  the  CDRL  distribution  and  DCMA  needs.  However, 
DCMA  could  not  provide  evidence  that  it  contacted  the  F-35  JPO  to  resolve  these 
CDRL  distribution  deficiencies.  DCMA  is  required  to  review  CDRLs  to  ensure 
that  they  perform  their  responsibilities  in  accordance  with  the  contract.  DCMA 
personnel  did  not  ensure  that  they  receive  all  necessary  CDRL  items,  which  may 
adversely  affect  their  ability  to  analyze,  identify,  and  take  action  to  prevent  or 
resolve  potential  and  existing  program  problems  throughout  all  program  phases. 
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DCMA  Instruction  1201,  " Corrective  Action  Process’’ 

DCMA  could  not  provide  sufficient  evidence  of  performing  activities  required 
in  DCMA  Instruction  1201,  "Corrective  Action  Process,"  for  15  of  the  16  CARs 
we  reviewed.  During  our  inspection,  we  did  not  find  supporting  information  to 
substantiate  activities  such  as  CAR  closure,  approval  of  contractor  corrective 
action  plans  for  CARs,  verification  of  corrective  actions  for  CARs,  follow  up  on 
Level  II  CARs,  and  escalation  of  CARs  to  the  next  higher  level  when  corrective 
actions  were  deemed  ineffective.  Without  sufficient  evidence  of  adherence 
to  the  corrective  action  process,  we  do  not  have  confidence  that  deficiencies 
were  captured,  completed,  and  verified  to  prevent  recurrence  of  deficiencies  in 
affected  products. 

Federal  Acquisition  Regulation  42.302,  "Contract  Administration  Functions" 
DCMA  could  not  provide  evidence  of  evaluating  and  making  recommendations 
for  the  acceptance  or  rejection  of  major  and  minor  waivers  and  deviations,  also 
referred  to  as  "variances."  According  to  FAR  42.302,  "Contract  Administration 
Functions,"  DCMA  is  to  assist  in  evaluating  and  make  recommendations  for 
acceptance  or  rejection  of  waivers  and  deviations.  The  MOA  between  the 
F-35  JPO  and  DCMA  did  not  relieve  DCMA  of  this  function.  The  MOA  stated, 

"This  MOA  is  intended  to  clarify  or  add  to  the  normal  Contract  Administration 
Services  (CAS)  functions  listed  in  the  Federal  Acquisition  Regulation  (FAR)  42.302 
and  Defense  Federal  Acquisition  Regulation  Supplement  (DFARS)  242.302."  The 
MOA  also  stated  that  DCMA  will  review  all  deviations  and  waivers  in  accordance 
with  the  F135  Configuration  Management  Plan.  DCMA  needs  to  perform  this 
function  to  ensure  that  waivers  and  deviations  are  correctly  classified,  reviewed, 
and  approved. 
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fFGUQFOverall 

Findings  and  Recommendations 

During  our  inspection,  we  noted  examples  where  the  F135  program  demonstrated 
sound  quality  assurance  practices.  We  were  impressed  with  Pratt  &  Whitney’s 
automated  processes,  which  were  implemented  to  reduce  product  variation 
due  to  human  factors  and  simplify  operational  task  complexity.  We  noted 
that  shop  floor  practices  were  well-controlled  with  the  use  of  electronic  work 
instructions  that  could  prevent  out-of-sequence  work.  In  design  and  development, 
we  noted  well-planned  processes  that  implemented  software  applications  that 
ensured  consistent  and  controlled  workflow.  In  supplier  quality,  we  found  that 
Pratt  &  Whitney  required  its  suppliers  to  be  AS9100  certified. 

F135  program  personnel  were  knowledgeable  of  their  respective  areas  and 
recognized  the  need  for  improvements.  However,  we  identified  61  nonconformities 
that  were  violations  of  AS9100C,  regulatory  requirements,  and  DoD  policies. 

Based  on  those  nonconformities,  we  identified  the  following  systemic  findings: 

•  additional  program  management  oversight  required, 

•  critical  safety  item  program  not  fully  compliant, 

•  contractor  shortfalls  in  continuous  improvement, 

•  insufficient  risk  management  practices, 

•  insufficient  supplier  selection  criteria  and  management  of 
underperforming  suppliers,  and 

•  inadequate  software  quality  management  practices. 

The  following  sections  discuss  our  overall  findings  and  recommendations. 
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Finding  A 

Additional  Program  Management  Oversight  Required 

We  found  that  additional  program  management  oversight  is  required  by  F-35  JPO 
and  DCMA,  as  evidenced  by  the  61  nonconformities  (violations  of  AS9100C, 
regulatory  requirements,  and  DoD  policy)  that  we  documented  during  our 
inspection.  F-35  JPO  and  DCMA  oversight  is  required  to  prevent  adverse  program 
cost,  schedule,  and  performance  issues. 


Discussion 

The  F-35  JPO  awarded  SDD  and  LRIP  contracts  with  CDRL  items  that  were 
delivered  for  information  only,  and  not  for  approval.  We  also  found  that  DCMA 
was  not  notifying  the  procuring  contracting  officer  of  their  absence  in  the 
distribution  list  for  CDRLs  impacting  program  surveillance  activities.  The  absence 
of  F-35  JPO  approval  and  DCMA  review  of  CDRL  items  limits  the  ability  of  the 
government  to  provide  clear  direction  and  guidance  to  the  contractor. 

The  F-35  JPO  used  the  variance  process  to  waive  requirements  for  extended 
periods  of  time,  which  was  a  misapplication  of  the  variance  request  process  and 
did  not  discourage  the  repeated  tender  of  nonconforming  supplies  or  services 
in  accordance  with  FAR  46.407,  "Nonconforming  Supplies  or  Services."  To  date, 
twenty  percent  of  the  original  175  MVRs  from  the  production  baseline  remain 
open;  approximately  two  percent  could  lead  to  specification  changes  to  meet  the 
performance  limitations.  The  F-35  JPO  will  continue  to  accept  nonconforming 
engines  if  MVR  extensions  continue  to  be  approved  and  not  resolved. 

DCMA  could  not  provide  evidence  of  evaluating  and  making  recommendations  for 
the  acceptance  or  rejection  of  waivers  and  deviations.  According  to  FAR  42.302, 
"Contract  Administration  Functions,"  DCMA  is  to  assist  in  evaluating  and  make 
recommendations  for  acceptance  or  rejection  of  waivers  and  deviations.  The  MOA 
between  the  F-35  JPO  and  DCMA  did  not  relieve  DCMA  of  this  function.  DCMA 
needs  to  perform  this  function  to  ensure  that  variances  are  correctly  classified, 
reviewed,  and  approved. 

DCMA  could  not  provide  sufficient  evidence  of  supporting  information  to 
substantiate  closure  of  CARs,  approval  of  contractor  corrective  action  plans  for 
CARs,  verification  of  corrective  actions  for  CARs,  follow-up  on  Level  II  CARs,  and 
escalation  of  CARs  to  the  next  higher  level  when  corrective  actions  were  deemed 
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ineffective.  Without  sufficient  evidence  of  adherence  to  the  corrective  action 
process,  we  do  not  have  confidence  that  deficiencies  were  captured,  completed,  and 
verified  to  prevent  recurrence  of  deficiencies  in  affected  products. 

Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  Adequacy  of  Program 
Management  Oversight 

The  F-35  JPO  disagreed  and  stated  that  adequate  program  management  oversight  is 
in  place.  However,  the  F-35  JPO  will  address  specific  issues  as  it  deems  necessary 
to  prevent  adverse  program  cost,  schedule,  and  performance  issues. 

Our  Response 

Our  report  identifies  gaps  in  program  management  oversight  of  the  F135  quality 
management  system.  Specifically,  nonconformities  were  documented  in  the 
areas  of:  risk  management,  CDRL  reviews,  critical  safety  item  program,  supplier 
management,  and  establishing  program  quality  assurance  goals  and  objectives. 

By  implementing  our  recommendations  to  conduct  an  effective  root  cause  analysis 
and  implement  corrective  actions  for  all  61  nonconformities  (violations  of  AS9100C, 
regulatory  requirements,  and  DoD  policies),  the  F-35  JPO  will  address  most  of  the 
program  management  oversight  shortfalls  that  we  found. 

Defense  Contract  Management  Agency  Comments  on  Review  of 
Contract  Deliverables 

DCMA  agreed  that  the  DCMA  Aircraft  Propulsion  Ops  (APO)  at  Pratt  &  Whitney 
did  not  review  all  CDRLs  to  identify  deliverables  that  impact  DCMA  program 
surveillance.  DCMA  believes  this  risk  was  mitigated  based  on  access  to 
Pratt  &  Whitney’s  electronic  document  storage  system  where  many  of  the 
CDRL  items  can  be  found  and  used  for  program  surveillance.  Going  forward, 

DCMA  APO-Pratt  &  Whitney  will  update  contract  receipt  and  review  process 
checklists  to  ensure  DCMA  is  included  in  the  distribution  of  all  CDRLs. 

Defense  Contract  Management  Agency  Comments  on  Review  of  Minor 
Nonconformances,  and  Waivers  and  Deviations 

DCMA  partially  agreed  and  stated  that  the  F135  LRIP  6  and  7  contracts  grant 
Pratt  &  Whitney  Material  Review  Board  (MRB)  authority  for  disposition  of  minor 
nonconformances  without  Government  approval.  The  F135  MOA  with  the  JPO 
further  states: 
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DCMA  will  monitor  minor  waivers  at  the  Pratt  &  Whitney  Material 
Review  Board  (MRB)  by  surveillance  of  the  Pratt  &  Whitney  MRB 
activities,  if  allowed  in  the  contract.  If  DCMA  questions  the  clarity 
or  technical  content  of  a  MRB  action,  DCMA  will  request  PW  [Pratt  & 

Whitney]  to  clarify  with  supporting  documentation  and  DCMA  may 
discuss  the  condition  with  the  JPO's  responsible  IPT  Lead. 

DCMA  APO-Pratt  &  Whitney  also  has  the  right  to  reject  engines  at  final  inspection 
when  the  Government  finds  fault  with  Pratt  &  Whitney’s  MRB  determination 
through  post-review.  DCMA  APO-Pratt  &  Whitney  will  review  and  update 
the  MOA  with  the  F-35  JPO  to  clarify  DCMA's  functions  for  evaluating  waivers 
and  deviations. 

Defense  Contract  Management  Agency  Comments  on  Corrective 
Action  Information 

DCMA  partially  agreed  and  stated  that  although  DCMA  could  not  provide  all 
requested  corrective  action  data  within  the  timeframe  requested,  supporting 
documentation  was  available.  DCMA  is  working  on  a  central  repository  to 
maintain  corrective  action  documentation  to  ensure  it  is  more  readily  accessible 
and  retrievable  in  the  future.  DCMA  agreed  that  it  did  not  comply  with  DCMA 
Instruction  1201,  and  the  Instruction  is  currently  under  revision  to  clarify  the 
CAR  elevation  process.  Upon  policy  update  approval,  DCMA  APQ-Pratt  &  Whitney 
will  review  all  open  CARS  to  determine  if  any  warrant  elevation  to  the  next  level, 
as  recommended.  DCMA  APO-Pratt  &  Whitney  will  assess  all  open  CARs  based  on 
the  new  policy  to  identify,  analyze,  and  assess  any  impacts  to  F-35  product. 

Our  Response 

We  appreciate  DCMA’s  plans  to  update  its  CDRL  review  process  and  develop  a 
central  repository  to  maintain  corrective  action  documentation.  However,  DCMA’s 
response  on  minor  nonconformances  and  waivers  and  deviations  do  not  align 
with  Federal  procurement  policy  on  accepting  or  rejecting  contractor  products  or 
services.  We  contacted  the  Defense  Procurement  and  Acquisition  Policy  (DPAP) 
under  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (OUSD(AT&L))  for  the  Department’s  official  perspective  on  FAR  46.407, 
"Nonconforming  supplies  or  services.”  DPAP  referenced  the  Office  of  Federal 
Procurement  Policy  (OFPP)  Policy  Letter  11-01,  "Performance  of  Inherently 
Governmental  and  Critical  Functions,"  which  states  that  accepting  or  rejecting 
contractor  products  or  services  is  an  inherently  governmental  function. 
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Recommendations,  Management  Comments, 
and  Our  Response 

Recommendation  A.l 

We  recommend  that  the  F-35  Joint  Program  Office: 

a.  Coordinate  with  the  Defense  Contract  Management  Agency  to  conduct 
an  effective  root  cause  analysis  and  implement  corrective  actions  for  all 
61  nonconformities  (violations  of  AS9100C,  regulatory  requirements,  and 
DoD  policies)  identified  during  our  inspection. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  coordinate  with  DCMA  to  conduct 
an  effective  root  cause  analysis  and  implement  corrective  actions  for  all 
61  nonconformities  by  the  end  of  December  2015. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation. 

No  further  comments  are  required. 

b.  Review  the  contract  data  requirements  list  and  determine  specific  items 
that  should  require  approval. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  review  CDRLs  and  determine  specific 
items  that  should  require  approval.  The  F-35  JPO  also  stated  that  it  reviewed  CDRL 
approval  determination  every  year  while  preparing  the  annual  LRIP  request  for 
proposal.  The  F-35  JPO  committed  to  complete  actions  for  this  recommendation  by 
the  end  of  April  2015. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation.  No 
further  comments  are  required. 

c.  Evaluate  open  major  variance  requests  to  determine  whether 
specification  changes  are  required  and  if  achievable  closure  plans  can 
be  developed. 
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F-35  Joint  Program  Office  Comments 

The  F-35  JPO  disagreed  and  stated  that  its  process  is  compliant.  The  F-35  JPO  also 
stated  that  the  F135  program  is  still  in  development  and  is  working  to  either  meet 
or  change  the  requirements  at  the  end  of  SDD.  All  F135  MVRs  have  achievable 
closure  plans  in  accordance  with  the  F-35  acquisition  strategy. 

Our  Response 

Comments  from  the  F-35  JPO  did  not  address  the  specifics  of  the  recommendation. 
According  to  FAR  46.407,  the  contracting  officer  must  discourage  the  repeated 
tender  of  nonconforming  supplies  or  services.  A  review  of  MVRs  for  LRIP  7 
indicates  that  the  program  office  is  extending  the  effectivity  of  existing  MVRs 
to  cover  entire  LRIP  7  units,  even  though  for  some  of  the  MVRs,  an  engineering 
solution  has  been  identified.  Concurrent  development  should  not  prevent  the 
F-35  JPO  from  closing  those  MVRs  not  associated  with  aircraft  system-level 
qualification  testing  and  those  MVRs  that  could  lead  to  specification  changes  to 
meet  performance  limitations  before  the  end  of  SDD.  In  addition,  engineering 
documentation  changes  should  be  made  to  identify  those  components  found  to  have 
limited  life  limitations.  We  request  that  the  F-35  JPO  provide  additional  comments 
in  response  to  the  final  report. 

Recommendation  A. 2 

We  recommend  that  the  Defense  Contract  Management  Agency: 

a.  Review  the  contract  data  requirements  list  items  to  identify  any 

deliverables  that  impact  Defense  Contract  Management  Agency’s  program 
surveillance  and  coordinate  with  the  F-35  Joint  Program  Office  to  resolve 
contract  data  requirements  list  distribution  issues. 

Defense  Contract  Management  Agency  Comments 

DCMA  agreed  and  stated  that  DCMA  APQ-Pratt  &  Whitney  will  ensure  CDRLs 
are  reviewed  as  part  of  the  contract  receipt  and  review  process  to  ensure  CDRLs 
required  for  program  surveillance  are  identified  and  coordinated  with  the  F-35  JPO 
and  the  contractor  to  ensure  proper  distribution  to  DCMA  APO-Pratt  &  Whitney. 
Contract  receipt  and  review  process  checklists  will  be  updated  to  ensure  DCMA  is 
included  in  the  distribution  of  all  CDRLs. 

Our  Response 

The  comments  from  DCMA  meet  the  intent  of  our  recommendation.  No  further 
comments  are  required. 
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b.  Review  and  update  the  Memorandum  of  Agreement  with  the  F-35  Joint 
Program  Office  to  ensure  that  all  Defense  Contract  Management  Agency 
functions  to  evaluate  waivers  and  deviations  are  clear  and  in  accordance 
with  the  Federal  Acquisition  Regulation. 

Defense  Contract  Management  Agency  Comments 

DCMA  agreed  and  stated  that  DCMA  APO  Pratt  &  Whitney  will  review  and  update 
the  MOA  with  the  F-35  JPO  to  clarify  DCMA 's  functions  for  evaluating  waivers  and 
deviations  and  ensuring  they  are  clear  and  in  accordance  with  the  FAR. 

Our  Response 

The  comments  from  DCMA  meet  the  intent  of  our  recommendation.  No  further 
comments  are  required  but  the  update  to  the  MOA  should  incorporate  additional 
information  that  we  received  on  FAR  46.407.  We  contacted  DPAP  for  the 
Department's  official  perspective  on  FAR  46.407.  DPAP  referenced  OFPP  Policy 
Letter  11-01,  which  states  that  accepting  or  rejecting  contractor  products  or 
services  is  an  inherently  governmental  function.  DPAP  further  stated  that, 

Since  acceptance  of  minor  non-conformances  is  an  inherently 
governmental  function,  delegation  should  not  be  allowed.  Contractor 
tasks  in  this  area  should  be  limited  to  assisting  Government 
personnel  in  evaluating  and  making  recommendations  to  accept  or 
reject  minor  non-conformances. 

c.  Ensure  that  all  associated  data  required  to  substantiate  corrective  action 
closure  is  accessible,  available,  and  retrievable. 

Defense  Contract  Management  Agency  Comments 

DCMA  partially  agreed  and  stated  that  although  DCMA  could  not  provide  all 
requested  corrective  action  data  within  the  requested  timeframe,  documentation  is 
available.  DCMA  is  working  on  a  central  repository  to  maintain  corrective  action 
documentation  to  ensure  it  is  more  readily  accessible,  available,  and  retrievable. 

Our  Response 

Although  data  were  available  during  the  inspection,  we  agree  with  DCMA's 
approach  to  create  a  central  repository  to  maintain  corrective  action 
documentation  to  ensure  it  is  more  readily  accessible,  available,  and  retrievable. 

No  further  comments  are  required. 
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Finding  B 

Critical  Safety  Item  Program  Not  Fully  Compliant 

The  F135  CSI  program  did  not  meet  DoD  CSI  requirements.  In  addition, 

Pratt  &  Whitney  did  not  meet  CSI  contractual  requirements.  A  CSI  program 
that  is  not  fully  compliant  with  DoD  policy  inhibits  the  ability  of  F135  personnel 
responsible  for  design,  development,  production,  test,  maintenance,  and  inspection 
to  perform  effective  CSI  life  cycle  management. 


Discussion 

The  F-35  JPO  required  Pratt  &  Whitney  to  meet  CSI  SOW  requirements  that 
were  based  on  NAVAIR  Instruction  4200. 25D,  dated  June  20,  2002,  instead  of 
the  Joint  Safety  Service  CSI  Instruction,  SECNAVINST  4140.2,  "Management  of 
Aviation  Critical  Items,"  dated  January  25,  2006.  Nevertheless,  we  found  that 
the  CSI  program  did  not  meet  either  NAVAIR  Instruction  4200. 25D  or  Joint 
Service  CSI  Instruction  in  several  areas.  Areas  where  the  F135  CSI  program  did 
not  meet  DoD  requirements  include  parts  identification,  critical  characteristic 
identification,  part  determination  methodology,  and  supplier  identification.  The 
lack  of  compliance  with  the  Joint  Service  CSI  Instruction  increases  the  risk 
of  nonconforming  CSI  parts  being  installed  on  aircraft,  which  directly  affects 
personnel  safety. 

Pratt  &  Whitney  processes  and  specifications  did  not  meet  all  CSI  contractual 
requirements  identified  in  the  CSI  SOW.  The  CSI  SOW  required  that  all  CSI  key 
product  characteristics  to  be  identified  and  designated  as  "critical"  or  "major." 
However,  Pratt  &  Whitney  designated  parts  on  the  CSI  list  with  undetermined  key 
product  characteristics.  Also,  Pratt  &  Whitney  did  not  ensure  that  all  drawings 
and  associated  technical  data  clearly  identify  that  an  item  was  CSI.  As  a  result, 
DCMA  is  unable  to  provide  source  inspection  on  all  CSI  parts  as  required. 
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Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  Critical  Safety  Item 
Program  Compliance 

The  F-35  JPO  partially  agreed  and  stated  that  CSI  requirements  were  flowed  to  the 
contractor  for  LRIP1  at  which  time  the  F-35/F135  CSJ  requirements  were  still  being 
formulated.  Beginning  with  LRIP  9,  the  F-35  JPO  will  include  the  part  number  also. 
The  F-35  JPO  is  working  to  align  with  updated  DoD  requirements. 

Our  Response 

The  F-35  JPO’s  plan  to  align  with  updated  DoD  CSI  instructions  will  address  our 
recommendations  on  the  F135  CSI  program. 

Recommendations,  Management  Comments, 
and  Our  Response 

Recommendation  B.l 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  critical  safety  item 
contractual  requirements  and  Pratt  &  Whitney’s  critical  safety  item  program 
processes  and  specifications  meet  the  intent  of  the  Joint  Critical  Safety  Item 
Instruction  and  supplemental  guidance  of  the  Joint  Aeronautical  Commanders 
Group  Aviation  Critical  Safety  Item  Management  Handbook. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  partially  agreed  and  stated  that  CSI  requirements  were  flowed  to 
the  contractor  for  LRIP1  at  which  time  the  F-35/F135  CSI  requirements  were  still 
being  formulated.  F135  CSI  parts  are  identified  by  nomenclature,  not  part  number. 
However,  beginning  with  LRIP  9  the  F-35  JPO  will  include  the  part  number  also. 

The  F-35  JPO  is  working  to  align  with  updated  DoD  requirements  and  also  working 
with  the  contractor  to  document  the  process  to  fully  align  with  DoD  requirements 
by  May  2015. 

Our  Response 

The  F-35  JPO’s  approach  to  align  with  updated  DoD  requirements  meet  the  intent 
of  our  recommendations.  However,  we  want  to  reiterate  that  the  scope  of  our 
recommendation  is  broader  than  just  addressing  nonconformities  with  part  number 
CSI  requirements.  No  further  comments  are  required. 
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Recommendation  B.2 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  critical  safety  items 
for  already  delivered  engines  meet  DoD  critical  safety  item  requirements  such  that 
critical  safety  items  receive  proper  care  and  handling  during  installation,  testing, 
and  field  maintenance. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  disagreed  and  stated  that  as  it  is  executing  actions  stated  for 
Recommendation  B.l,  if  the  F-35  JPO  discovers  that  a  significant  escape  has 
occurred,  then  it  will  take  appropriate  action  for  delivered  engines. 

Our  Response 

The  F-35  JPO’s  approach  to  take  appropriate  action  based  on  results  of 
implementing  recommendation  B.l  meet  the  intent  of  our  recommendation. 
However,  upon  alignment  with  updated  DoD  CSI  instructions,  we  request 
that  the  F-35  JPO  provide  us  with  evidence  that  delivered  engines  will  meet 
DoD  CSI  requirements. 
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{FOWOJ-Finding  C 

Contractor  Shortfalls  in  Continuous  Improvement 

(FOUO)  The  F-35  JPO  did  not  establish  F135  program  quality  goals  and  objectives 
that  were  mutually  agreed  upon  by  Pratt  &  Whitney  for  current  contracts. 
Additionally,  Pratt  &  Whitney  metrics  did  not  show  improvement  in  quality 
assurance,  process  capability,  and^^^^^^^|  If  the  F135  program  does  not 
make  continuous  improvement  a  top  priority,  product  quality  could  degrade, 
especially  with  increasing  production  quantities  over  the  next  several  years. 


(FOUO)  Discussion 

There  were  no  established  program  objectives  that  were  mutually  agreed 
upon  between  Pratt  &  Whitney  and  F-35  JPO  for  the  SDD  contract.  In  addition, 
Pratt  &  Whitney  did  not  have  an  established  F135  quality  plan  that  describes 
unique  F135  program  quality  assurance  requirements.  Pratt  &  Whitney  formally 
requested  relief  to  eliminate  the  requirement  to  establish  a  quality  plan  for  the 
F135  program  on  the  PSDP;  however,  the  F-35  JPO  did  not  provide  Pratt  &  Whitney 
formal  relief  from  this  requirement.  Pratt  &  Whitney  has  an  enterprise-level 
quality  manual;  however,  this  quality  manual  did  not  meet  the  SOW  requirements 
for  technical  performance  measures,  product  assurance  requirements  and  goals, 
and  critical  manufacturing  and  supporting  processes. 

(FOUO)  We  documented  several  nonconformities  with  Pratt  &  Whitney’s 
performance  measures.  For  example,  Pratt  &  Whitney  did  not  reduce 
nonconforming  items  per  engine  and  quality  escapes,  and 

Additionally,  Pratt  &  Whitney  did  not 

meet  process  capability  (Cpk)  goals  and  did  not  consistently  analyze  and  report 
KPC  Cpk  data  for  F135  engine  hardware.  DCMA  relied  on  this  data  to  establish 
mandatory  Government  inspection  points.  The  F-35  JPO  also  monitored  the  data  as 
an  indication  of  technical  and  program  risk.  If  Pratt  &  Whitney  personnel  do  not 
correctly  analyze  and  report  KPC  data,  then  the  F-35  JPO  and  DCMA  cannot  make 
sound  program  and  resource  management  decisions. 
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Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  the  F135  Quality  Plan 
The  F-35  JPO  partially  agreed  and  stated  that  it  accepted  the  Pratt  &  Whitney 
Company  Quality  Plan  instead  of  a  specific  F135  Quality  Plan.  The  F-35  JPO  also 
stated  that  there  is  neither  a  Department  nor  other  defined  requirement  for  a 
program-specific  quality  assurance  plan.  In  addition,  it  is  not  required  by  the  SOW. 
The  program  manager  has  the  responsibility  to  decide  how  to  proceed  based  on  the 
size/complexity/risk  of  the  program. 

Our  Response 

The  F-35  JPO’s  plan  to  ensure  that  Pratt  &  Whitney’s  quality  plan  meets  contractual 
requirements  will  address  our  recommendation. 

Recommendations,  Management  Comments, 
and  Our  Response 

Recommendation  C.l 

We  recommend  that  the  F-35  Joint  Program  Office  establish  F135  program  quality 
goals  and  objectives  that  are  mutually  agreed  upon  by  Pratt  &  Whitney  and  track 
Pratt  &  Whitney’s  performance  against  those  objectives. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  partially  agreed  and  stated  that  program  quality  goals  and  objectives 
have  been  established  and  agreed  upon.  They  are  tracked  and  presented  to 
program  management.  Since  the  PEOs  November  2014  Quality  Update,  the  program 
has  established  F135-specific  program  goals.  The  F-35  JPO  plans  to  reflect  quality 
goals  and  objectives  in  future  production  contracts. 

Our  Response 

Although  program  goals  and  objectives  were  not  in  place  at  the  time  of  our 
inspection,  we  are  pleased  that  the  F-35  JPO  has  since  established  them  in  the 
PEO’s  November  2014  Quality  Update  and  are  planning  to  reflect  those  goals  and 
objectives  in  future  production  contracts.  We  request  that  the  F-35  JPO  provide  us 
with  the  updated  program  quality  goals  and  objectives  to  determine  if  it  meets  the 
intent  of  our  recommendation. 
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Recommendation  C.2 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  & 
Whitney’s  quality  plan  meets  contractual  requirements. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  ensure  that  Pratt  &  Whitney’s  quality 
plan  meets  contractual  requirements  as  reflected  in  future  production  contracts. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation. 

No  further  comments  are  required. 

Recommendation  C.3 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney 
consistently  analyzes  and  reports  KPC  Cpk  data  for  F135  engine  hardware  and  that 
performance  improvement  plans  are  established. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  ensure  that  Pratt  &  Whitney 
consistently  analyzes  and  reports  KPC  Cpk  data  for  F135  engine  hardware  and 
that  performance  improvement  plans  are  established.  The  F-35  JPO  committed  to 
complete  actions  for  this  recommendation  by  April  2015. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation. 

No  further  comments  are  required. 
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Finding  D 

Insufficient  Risk  Management  Practices 

The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney  proactively  identify,  elevate, 
track,  and  manage  F135  program  risks,  in  accordance  with  the  F135  risk 
management  plan.  As  a  result,  risks  that  can  adversely  affect  the  F135  program 
may  not  receive  program  management  visibility  and  awareness. 


Discussion 

The  risk  management  program  at  Pratt  &  Whitney  did  not  provide  program 
management  visibility  and  awareness  of  risks  that  can  adversely  affect  the 
program.  We  found  that  the  RIMB  only  discussed  technical  risks  and  not 
production,  sustainment,  design,  or  supply  chain  risks.  There  was  also  insufficient 
evidence  that  the  required  personnel  were  present  during  the  RIMB  to  make 
decisions  on  risks.  Lastly,  we  found  that  the  Pratt  &  Whitney’s  F135  risk 
management  database  did  not  reflect  accurate  risk  information  including  risk 
category,  status,  mitigation  plans,  and  risk  ranking. 

Pratt  &  Whitney  generated  issues  at  the  software  development  integrated  product 
team  level;  however,  there  was  no  evidence  that  these  issues  were  analyzed  to 
determine  system  level  risks.  For  example,  there  were  at  least  40  open  safety 
critical  technical  issues  without  risk  analysis  documentation  to  determine  whether 
the  issues  were  potential  software  risks  that  should  be  elevated  to  the  RIMB. 

To  effectively  perform  risk  management  activities,  the  DOD  risk  management  guide 
recommends  proactive  identification  and  analyses  of  risks  during  all  program 
phases;  effective  implementation  of  corrective  action;  continuous  monitoring;  and 
communication,  documentation,  and  coordination. 

The  F-35  JPO  should  ensure  that  Pratt  &  Whitney  maintain  an  effective  risk 
management  program  with  rigor  and  discipline  to  identify,  evaluate,  and  mitigate 
risks;  otherwise,  significant  risks  and  issues  that  should  be  managed  and  tracked  at 
the  system  level  may  be  overlooked. 
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Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  Risk  Management 
The  F-35  JPO  disagreed  and  stated  that  Pratt  &  Whitney  follows  standard  risk 
management  processes  for  identification  and  tracking.  The  F-35  JPO  also  stated 
that  high-level  risks  are  discussed  by  the  air  system  management  teams  at 
regular  meetings. 

Our  Response 

During  our  inspection  we  found  nonconformities  with  Pratt  &  Whitney’s  lack  of 
adherence  to  the  F135  risk  management  process.  For  example,  we  found  that  the 
RIMB  only  discussed  technical  risks,  the  risk  management  database  was  incomplete 
and  inaccurate,  and  there  was  insufficient  evidence  that  lower-level  risks  were 
analyzed  and  elevated. 

Recommendation,  Management  Comments, 
and  Our  Response 

Recommendation  D 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney 
identify,  elevate,  track,  and  manage  all  risks  that  affect  the  program,  including 
software  and  supply  chain  risks. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  partially  agreed  and  stated  that  it  will  ensure  Pratt  &  Whitney 
identify,  elevate,  track,  and  manage  all  risks  that  affect  the  program,  including 
software  and  supply  chain  risks.  The  F-35  JPO  also  clarified  that  the  F135  RIMB 
is  not  the  only  process  used  and  that  software  system-level  risks  are  already 
evaluated  and  documented  at  the  software  integrated  product  team  board  weekly 
meeting.  Items  with  significant  air  system-level  impact  also  receive  an  Air  System 
Software  Problem  Anomaly  Report  (SPAR)  Board  severity  assessment.  Issues 
with  a  high  criticality  rank,  which  impact  safety  or  major  program  objectives,  are 
treated  properly  by  elevating  to  program  management  and  taking  immediate  action 
to  fix  the  problem. 
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Our  Response 

The  F-35  JPO’s  commitment  to  ensure  that  Pratt  &  Whitney  identifies,  elevates, 
tracks,  and  manages  all  risks  that  affect  the  program  meet  the  intent  of  our 
recommendations.  However,  at  the  time  of  our  inspection,  we  found  that  the  risk 
management  database  at  Pratt  &  Whitney  was  incomplete  and  inaccurate,  and 
there  was  insufficient  evidence  that  the  lower  component-level  software  risks  were 
identified,  elevated,  tracked,  and  managed.  No  further  comments  are  required. 
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Finding  E 

Insufficient  Supplier  Selection  Criteria  and 
Management  of  Underperforming  Suppliers 

The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney’s  supplier  selection  criteria 
and  management  of  underperforming  suppliers  were  sufficient.  As  a  result, 
Pratt  &  Whitney  cannot  provide  the  F-35  JPO  assurance  that  their  suppliers  are 
qualified  to  deliver  quality  products. 


Discussion 

Pratt  &  Whitney’s  minimum  baseline  criteria  for  selecting  suppliers,  such 
as  performance/quality  history,  and  cost  and  schedule  data,  were  not  well 
defined  within  its  procurement  policies  and  procedures.  We  also  found  that 
Pratt  &  Whitney  did  not  adequately  define  the  actions  that  should  be  taken  when 
a  supplier  continues  to  have  a  poor  quality  rating  or  is  considered  a  high  risk.  We 
identified  multiple  F135  suppliers  that  were  considered  high  risk  for  extended 
periods  of  time.  One  supplier  was  rated  as  high  risk  for  33  consecutive  months. 

Pratt  &  Whitney  did  not  review  and  verify  CARs  issued  to  its  suppliers  in  a  timely 
manner.  In  many  cases,  the  supplier  responded  with  a  root  cause  corrective  action 
plan  by  the  due  date;  however,  Pratt  &  Whitney  did  not  review  and  close  out  the 
CARs  within  a  reasonable  timeframe.  Some  CARs  were  not  closed  out  until  almost 
2  years  later.  If  CARs  are  not  closed  out  in  a  timely  manner,  then  there  is  a  high 
risk  of  continued  recurrence  of  the  nonconformance. 

Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  Supplier  Management 
F-35  JPO  partially  agreed  and  stated  that  supplier  selection  was  sufficient;  however, 
it  needs  to  re-evaluate  Pratt  &  Whitney’s  management  of  underperforming 
suppliers  to  ensure  suppliers  are  qualified  to  deliver  quality  products. 

Our  Response 

The  F-35  JPO’s  plan  to  work  with  DCMA  to  ensure  that  Pratt  &  Whitney  implements 
criteria  for  supplier  selection  and  actions  to  be  taken  for  suppliers  that  continue  to 
be  high  risk  will  address  our  recommendations. 
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Recommendations,  Management  Comments, 
and  Our  Response 
Recommendation  E.l 

We  recommend  that  the  F-35  Joint  Program  Office  work  with  the  Defense  Contract 
Management  Agency  to  ensure  that  Pratt  &  Whitney  clearly  defines,  documents, 
and  implements  minimum  baseline  criteria  for  supplier  selection  and  actions  to  be 
taken  for  suppliers  that  continue  to  be  high  risk. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  work  with  the  DCMA  to  ensure  that 
Pratt  &  Whitney  clearly  defines,  documents,  and  implements  minimum  baseline 
criteria  for  supplier  selection  and  actions  to  be  taken  for  suppliers  that  continue  to 
be  high  risk. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation. 

No  further  comments  are  required. 

Recommendation  E.2 

We  recommend  that  the  Defense  Contract  Management  Agency  perform  additional 
surveillance  on  Pratt  &  Whitney’s  corrective  action  requests  issued  to  suppliers  to 
ensure  that  they  are  closed  within  a  reasonable  timeframe. 

Defense  Contract  Management  Agency  Comments 

DCMA  agreed  and  stated  that  DCMA  APO-Pratt  &  Whitney  will  incorporate  supplier 
corrective  action  into  the  quality  management  system  surveillance  plan  to  ensure 
Pratt  &  Whitney  management  of  supplier  corrective  action  is  in  compliance  with  its 
quality  system. 

Our  Response 

The  comments  from  DCMA  meet  the  intent  of  our  recommendation.  No  further 
comments  are  required. 
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Finding  F 

Inadequate  Software  Quality  Management  Practices 

The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney  demonstrated  adequate  software 
quality  management  practices.  Pratt  &  Whitney  had  an  obsolete  SDP  and  had 
requirements  that  were  not  fully  implemented  or  traceable.  Additionally,  Pratt  & 
Whitney  was  not  performing  required  audits  and  reviews  to  ensure  that  software 
products  meet  contractual  requirements.  As  a  result,  there  is  no  assurance  that 
software  products  meet  contractual  requirements  and  there  is  increased  risk  to 
performance  of  critical  software. 


Discussion 

The  F135  SDP  did  not  describe  the  current  software  architecture,  did  not 
identify  all  software  configuration  items,  and  did  not  define  all  applicable 
PSLs.  In  addition,  boot  CSCI  software  requirements  were  not  implemented  or 
traceable  to  the  software  design  description,  the  software  test  plan,  and  the 
software  test  description.  For  example,  there  was  no  evidence  that  the  safety 
requirement  for  the  boot  software  to  provide  fail-safe  recovery  from  inadvertent 
instruction  jumps  was  met.  Pratt  &  Whitney  was  also  not  developing  and  testing 
PHM  software  at  the  same  PSL  of  the  CSCI  that  it  is  embedded  in.  Incomplete 
requirements  verification  and  software  not  developed  and  tested  in  accordance 
with  PSL  requirements  could  create  a  performance  and  safety  risk  for  critical 
F135  software. 

We  found  several  issues  with  Pratt  &  Whitney’s  SQA  organization.  For  example, 
the  SQA  organization  did  not  conduct  physical  configuration  audits  and  functional 
configuration  audits  for  the  main  engine,  lift  system,  and  boot  CSCIs.  For 
supplier-developed  software,  the  SQA  organization  did  not  conduct  software 
lifecycle  audits  on  all  software  suppliers.  Moreover,  there  was  no  evidence  that 
the  SQA  organization  verified  software  change  requests,  witnessed  testing  and 
integration  reviews,  and  reviewed  software  release  packages  in  accordance  with 
its  own  internal  procedure.  The  F-35  JPO  must  ensure  that  Pratt  &  Whitney’s  SQA 
organization  perform  its  required  functions  such  that  quality  assurance  practices 
are  being  implemented  for  critical  software  products. 
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Management  Comments  on  the  Finding 
and  Our  Response 

F-35  Joint  Program  Office  Comments  on  Software  Quality  Management 
The  F-35  JPO  disagreed  and  stated  that  software  quality  management  practices  are 
in  place,  and  technical  reviews  have  been  conducted.  The  F-35  JPO  also  stated  that 
the  software  development  plan  was  in  review  before  our  inspection. 

Our  Response 

Pratt  &  Whitney  has  processes  in  place;  however,  the  nonconformities  we 
documented  during  the  inspection  pertained  to  implementation.  For  example, 

Pratt  &  Whitney’s  SQA  organization  did  not  conduct  physical  configuration  audits, 
functional  configuration  audits,  and  software  lifecycle  audits,  and  had  no  evidence 
that  the  SQA  organization  verified  software  change  requests,  witnessed  testing,  or 
reviewed  software  release  packages.  F-35  JPO’s  response  did  not  provide  detail  to 
substantiate  its  disagreement  with  our  finding  discussions. 

Recommendations,  Management  Comments, 
and  Our  Response 

Recommendation  F.l 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney 
resolve  nonconformities  related  to  software  quality  management  systems, 
including  an  obsolete  F135  Software  Development  Plan,  lack  of  software 
requirements  traceability,  and  software  not  developed  to  the  appropriate  product 
software  level. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  disagreed  and  stated  that  software  is  developed  to  the  correct  PSL. 
The  F-35  JPO  also  stated  that  the  software  development  plan  is  not  obsolete  but 
needs  only  minor  updates  that  were  already  in  work  before  this  inspection.  These 
updates  do  not  invalidate  the  software  flight  clearances  completed  to  date  as  they 
merely  clarify  existing  processes  and  procedures. 
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Our  Response 

We  do  not  agree  that  that  the  software  development  plan  is  not  obsolete  but 
only  needs  minor  updates.  The  F135  SDP  did  not  describe  the  current  software 
architecture,  did  not  identify  all  software  configuration  items,  and  did  not 
define  all  applicable  PSL.  An  SDP  that  does  not  define  or  reference  all  software 
configuration  items  and  their  levels  of  control  can  lead  to  conflicting  requirements, 
incorrect  design  implementation,  and  rework. 

In  reference  to  whether  software  is  developed  to  the  correct  PSL,  we  found  that 
because  the  PHM  software  is  now  embedded  in  the  main  engine  and  lift  system, 
which  are  designated  at  the  highest  PSL,  PHM  should  be  developed  and  tested  to 
the  same  requirements,  unless  controls  are  in  place  to  ensure  it  won’t  adversely 
affect  the  main  engine  and  lift  system  software. 

Lastly,  the  F-35  JPO  did  not  provide  details  or  address  our  recommendation  on 
resolving  nonconformities  on  the  lack  of  software  requirements  traceability.  We 
request  that  the  F-35  JPO  reconsider  its  position  and  provide  additional  comments 
and  substantiating  information  in  response  to  the  final  report. 

Recommendation  F.2 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney’s 
Software  Quality  Assurance  organization  conduct  audits,  reviews,  and  verification 
activities  of  both  internally-developed  and  supplier-developed  software. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  agreed  and  stated  that  it  will  ensure  that  Pratt  &  Whitney’s  Software 
Quality  Assurance  organization  conduct  audits,  reviews,  and  verification  activities 
of  both  internally  developed  and  supplier  developed  software. 

Our  Response 

The  comments  from  the  F-35  JPO  meet  the  intent  of  our  recommendation. 

No  further  comments  are  required. 
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Recommendation  F.3 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  Pratt  &  Whitney  assess 
the  impact  of  insufficient  software  verification  on  delivered  engines. 

F-35  Joint  Program  Office  Comments 

The  F-35  JPO  disagreed  and  stated  that  the  F135  propulsion  system  software 
was  properly  tested  against  the  standards  in  the  software  development  plan  and 
work  instructions.  Multiple  flight  clearances  performed  since  the  beginning  of  the 
F135  program  are  supported  by  the  excellent  safety  record  for  software  in  engine 
ground  test,  flight  test,  and  production. 

Our  Response 

Comments  from  the  F-35  JPO  did  not  fully  address  the  specifics  of  the 
recommendation.  The  nonconformities  we  documented  during  our  inspection 
identified  that  there  were  software  requirements  for  the  Boot  CSCI  and  PHM  that 
were  not  tested  or  verified.  For  example,  there  was  no  evidence  that  the  safety 
requirement  for  the  boot  software  to  provide  fail-safe  recovery  from  inadvertent 
instruction  jumps  was  met.  We  do  not  dispute  the  safety  record  for  software  in 
engine  ground  test,  flight  test,  and  production;  however,  such  testing  does  not 
guarantee  that  all  software  requirements  and  functionality  have  been  verified.  We 
request  that  the  F-35  JPO  reconsider  its  position  and  provide  additional  comments 
in  response  to  the  final  report. 
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Appendix  A 

Scope  and  Methodology 

We  conducted  this  evaluation  from  October  2014  through  February  2015  in 
accordance  with  the  Council  of  Inspectors  General  on  Integrity  and  Efficiency, 
"Quality  Standards  for  Inspection  and  Evaluation."  Those  standards  require  that 
we  plan  and  perform  the  evaluation  to  obtain  sufficient,  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  evaluation 
objectives.  We  believe  that  the  evidence  obtained  provides  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  evaluation  objectives. 

We  performed  quality  assurance  inspections  at  the  F-35  JPO  in  Arlington,  Virginia, 
and  Pratt  &  Whitney  facilities  in  East  Hartford  and  Middletown,  Connecticut,  to 
evaluate  the  F135  quality  management  system.  We  also  met  with  DCMA  personnel 
directly  involved  with  the  F-35  engine  program.  Our  inspection  focused  on 
the  following: 

•  applicable  statutory  and  regulatory  requirements, 

•  contractual  quality  management  system  requirements  (AS9100C),  and 

•  internal  quality  assurance  processes  and  procedures. 

Statutory  and  Regulatory  Requirements 

We  inspected  the  F-35  JPO  and  Pratt  &  Whitney  implementation  of  aviation 
critical  safety  item  requirements.  Public  Law  108-136,  Section  802,  "Quality 
control  in  procurement  of  aviation  CSIs  and  related  services,"  requires  DoD  to 
prescribe  a  quality  control  policy  for  the  procurement  of  aviation  CSIs.  Title  10, 
Section  2319  of  the  United  Sates  Code  is  the  applicable  statute  that  implements 
aviation  CSI  requirements  of  Public  Law  108-136,  Section  802.  Joint  Service 
CSI  Instruction,  "Management  of  Aviation  Critical  Safety  Items,"  implements  the 
DOD  CSI  program  and  establishes  the  policies,  procedures,  and  responsibilities  to 
manage  CSI.  The  Joint  Aeronautical  Commanders  Group  (JACG),  "JACG  Aviation 
Critical  Safety  Item  Handbook,"  implements  the  policies  in  the  Joint  Service 
CSI  Instruction  and  describes  the  technical  and  quality  assurance  requirements  for 
a  prime  contractor  CSI  program. 

Additionally,  we  evaluated  the  compliance  of  applicable  requirements  in  the 
Federal  Acquisition  Regulation  and  DoD  Directive  5000.01,  "The  Defense 
Acquisition  System." 
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AS9100  Standard 

We  performed  our  inspection  against  the  contractually  required  AS9100C  standard. 

The  AS9100C  standard  breaks  down  quality  assurance  requirements  into 
five  major  clauses: 

•  Quality  Management  System, 

•  Management  Responsibility, 

•  Resource  Management, 

•  Product  Realization,  and 

•  Measurement,  Analysis,  and  Improvement. 

The  Quality  Management  System,  Management  Responsibility,  and  Resource 
Management  clauses  require  the  organization  to  have  a  quality  assurance 
management  organization  that  has  all  the  resources  and  authority  to  affect  the 
end-item  quality  of  the  product.  In  addition,  it  requires  the  organization  to  have 
a  quality  assurance  manual  and  strict  control  over  all  documentation,  data,  and 
procedures  that  affect  the  quality  of  the  product.  Product  Realization  covers  the 
activities  and  processes  necessary  to  bring  a  product  into  existence. 

Product  realization  is  broken  down  further  by  the  AS9100  standard  as  follows: 

•  Planning  of  Product  Realization, 

•  Customer-Related  Processes, 

•  Design  and  Development, 

•  Purchasing, 

•  Production  and  Service  Provision,  and 

•  Control  of  Monitoring  and  Measuring  Equipment. 

Planning  of  Product  Realization  requires  the  organization  to  develop  processes 
needed  for  design  and  development  of  product  and  includes  elements  such 
as  procedures,  quality  assurance  records,  resource  requirements,  safety  and 
reliability  programs,  and  inspection  and  test.  Customer-Related  Process  requires 
the  organization  to  determine  customer  requirements  both  specified  and  derived. 
These  requirements  include  technical,  statutory,  and  regulatory  requirements. 
Design  and  Development  includes  requirements  that  cover  planning,  inputs, 
outputs,  review,  verification,  validation,  and  control  of  changes  as  related  to 
design  and  development.  Purchasing  requires  the  organization  to  ensure  that 
the  purchased  product  conforms  to  specified  purchase  requirements  and  that 
all  products  purchased  from  suppliers  are  verified  against  these  requirements. 
Production  and  Service  Provision  requires  the  organization  to  ensure  that 
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production  is  accomplished  under  controlled  conditions  using  drawings  and 
specifications,  work  instructions,  production  tools  and  software  programs, 
monitoring  and  measuring  equipment,  and  evidence  that  all  production  and 
inspection/verification  operations  have  been  completed  as  planned.  Control  of 
Monitoring  and  Measuring  Equipment  requires  the  organization  to  ensure  that 
devices  used  for  determining  product  compliance  with  performance  characteristics 
are  properly  maintained  to  provide  assurance  of  credible  measurements. 

Measurement,  Analysis,  and  Improvement  requires  the  organization  to  ensure 
the  product  continuously  improves.  This  clause  includes  customer  satisfaction, 
internal  audit,  monitoring  and  measuring  processes  and  product,  and  control  of 
nonconforming  products  to  ensure  continual  improvement. 

Use  of  Computer-Processed  Data 

We  did  not  use  computer-processed  data  to  perform  this  evaluation. 

Use  of  Technical  Assistance 

We  established  multi-disciplined  teams  of  engineers,  military  aviation  CSI  subject 
matter  experts,  and  quality  assurance  subject  matter  experts  who  evaluated  the 
F-35  engine  program  based  on  the  AS9100C  quality  management  standard.  Team 
members  were  trained  and  certified  in  AS9100,  with  an  average  of  17  years  of 
quality  assurance  audit  experience.  Additionally,  our  team  included  two  subject 
matter  experts  in  military  aviation  CSI. 
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Appendix  B 

Prior  Coverage 

During  the  last  5  years,  the  Government  Accountability  Office  (GAO)  and  the 
Department  of  Defense  Office  of  Inspector  General  (DoD  OIG)  issued  29  reports 
discussing  the  F-35  Joint  Strike  Fighter.  Unrestricted  GAO  reports  can  be  accessed 
at  http://www.gao.gov.  Unrestricted  DoD  IG  reports  can  be  accessed  at 

http://www.dodig.mil/pubs/index.cfm. 

GAO 

Report  No.  GAO-14-778,  "F-35  Sustainment:  Need  for  Affordable  Strategy, 

Greater  Attention  to  Risks,  and  Improved  Cost  Estimates,"  September  23,  2014 

Report  No.  GAO-14-340SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  31,  2014 

Report  No.  GAO-14-468T,  "F-35  Joint  Strike  Fighter:  Slower  Than  Expected  Progress 
in  Software  Testing  May  Limit  Initial  Warfighting  Capabilities,"  March  26,  2014 

Report  No.  GAO-14-322,  "F-35  Joint  Strike  Fighter:  Problems  Completing  Software 
Testing  May  Hinder  Delivery  of  Expected  Warfighting  Capabilities,”  March  24,  2014 

Report  No.  GAO-13-690T,  "F-35  Joint  Strike  Fighter:  Restructuring  Has  Improved 
the  Program,  but  Affordability  Challenges  and  Other  Risks  Remain,"  June  19,  2013 

Report  No.  GAO-13-500T,  "F-35  Joint  Strike  Fighter:  Program  Has  Improved  in  Some 
Areas,  but  Affordability  Challenges  and  Other  Risks  Remain,”  April  17,  2013 

Report  No.  GAO-13-294SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,”  March  28,  2013 

Report  No.  GAO-13-309,  "F-35  Joint  Strike  Fighter:  Current  Outlook  Is  Improved,  but 
Long-Term  Affordability  Is  a  Major  Concern,"  March  11,  2013 

Report  No.  GAO-12-437,  "Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further 
Enhance  Restructuring  and  Address  Affordability  Risks,"  June  14,  2012 

Report  No.  GAO-12-400SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  29,  2012 

Report  No.  GAO-12-525T,  "Joint  Strike  Fighter:  Restructuring  Added  Resources  and 
Reduced  Risk,  but  Concurrency  Is  Still  a  Major  Concern,"  March  20,  2012 
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Report  No.  GAO-11-903R,  "Joint  Strike  Fighter:  Implications  of  Program 
Restructuring  and  Other  Recent  Developments  on  Key  Aspects  of  DOD’s  Prior 
Alternate  Engine  Analyses,"  September  14,  2011 

Report  No.  GAO-11-677T,  "Joint  Strike  Fighter:  Restructuring  Places  Program  on 
Firmer  Footing,  but  Progress  Is  Still  Lagging,”  May  19,  2011 

Report  No.  GAO-11-325,  "Joint  Strike  Fighter:  Restructuring  Places  Program  on 
Firmer  Footing,  but  Progress  Still  Lags,"  April  7,  2011 

Report  No.  GAO-11-450T,  "Joint  Strike  Fighter:  Restructuring  Should  Improve 
Outcomes,  but  Progress  Is  Still  Lagging  Overall,”  March  15,  2011 

Report  No.  GAO-11-323R,  "Tactical  Aircraft:  Air  Force  Fighter  Force  Structure 
Reports  Generally  Addressed  Congressional  Mandates,  but  Reflected  Dated  Plans 
and  Guidance,  and  Limited  Analyses,"  February  24,  2011 

Report  No.  GAO-11-171R,  "Defense  Management:  DOD  Needs  to  Monitor  and  Assess 
Corrective  Actions  Resulting  from  Its  Corrosion  Study  of  the  F-35  Joint  Strike 
Fighter,”  December  16,  2010 

Report  No.  GAO-10-1020R,  "Joint  Strike  Fighter:  Assessment  of  DOD’s  Funding 
Projection  for  the  F136  Alternate  Engine,"  September  15,  2010 

Report  No.  GAO-10-789,  "Tactical  Aircraft:  DOD’s  Ability  to  Meet  Future 
Requirements  is  Uncertain,  with  Key  Analyses  Needed  to  Inform  Upcoming 
Investment  Decisions,"  July  29,  2010 

Report  No.  GAO-10-388SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  30,  2010 

Report  No.  GAO-10-478T,  "Joint  Strike  Fighter:  Significant  Challenges  and  Decisions 
Ahead,"  March  24,  2010 

Report  No.  GAO-10-382,  "Joint  Strike  Fighter:  Additional  Costs  and  Delays  Risk  Not 
Meeting  Warfighter  Requirements  on  Time,”  March  19,  2010 

Report  No.  GAO-10-520T,  "Joint  Strike  Fighter:  Significant  Challenges  Remain  as 
DOD  Restructures  Program,”  March  11,  2010 

Report  No.  GAO-09-711T,  "Joint  Strike  Fighter:  Strong  Risk  Management  Essential  as 
Program  Enters  Most  Challenging  Phase,"  May  20,  2009 
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Report  No.  GAO-09-326SP,  "Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,"  March  30,  2009 

Report  No.  GAO-09-303,  "Joint  Strike  Fighter:  Accelerating  Procurement 
before  Completing  Development  Increases  the  Government’s  Financial  Risk," 

March  12,  2009 

DOD  OIG 

Report  No.  DODIG-2015-092,  "F-35  Lightning  II  Program  Quality  Assurance  and 
Corrective  Action  Evaluation,”  March  11,  2015 

Report  No.  DODIG-2013-140,  "Quality  Assurance  Assessment  of  the  F-35  Lightning  II 
Program,”  September  30,  2013 

Report  No.  DODIG-2013-031,  "Audit  of  the  F-35  Lightning  II  Autonomic  Logistics 
Information  System  (ALIS),"  December  10,  2012 
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Management  Comments 

F-35  Joint  Program  Office  and  Defense  Contract 
Management  Agency  Comments 


F-35  LIGHTNING  II  JOINT  PROGRAM  OFFICE 
200  I2,b  Street  South*  Suite  600 
Arlington,  Virginia  22202-5402 


WAR  n  2015 


From:  Program  Executive  Officer.  F-35  Lightning  11  Joint  Program  Office  (JPO)  and  Director. 
Defense  Contract  Management  Agency  (DCMA) 


To:  Department  of  Defense  (DoD)  Deputy  Inspector  General  (1G)  for  Policy  and  Oversight 


Subj:  RESPONSE  TO  F-35  ENGINE  QUALITY  ASSURANCE  INSPECTION  (PROJECT  NO, 
D20 1 4-  DTOT  AD-0003 .000) 


Thank  you  for  the  opportunity  to  comment  on  your  draft  report.  "F-35  Engine  Quality 
Assurance  Inspection”,  dated  25  February  2015. 


Both  the  F-35  Lightning  II  Joint  Program  Office  (JPO)  and  the  Defense  Contract 
Management  Agency  (DCMA)  take  very  seriously  the  findings  and  recommendations  offered  in 
the  report.  As  noted  in  Enclosure  l,  F-35  JPO  and  DCMA  Response  to  Findings  and 
Recommendations,  our  team  has  aggressively  addressed  DoD  1G  findings  since  receipt  of  the 
draft  report  in  January  2015,  The  implementation  of  these  corrective  actions  has  led  to 
significant  progress  in  delivering  a  quality,  timely,  and  cost-effective  weapon  system  to  the 
United  States  and  its  allies. 


Enclosure  t  provides  a  detailed  response  to  each  finding  and  recommendation. 
The  action  officers  for  this  response  are 


CHRISTOPHER. T.  BOGDAN 
Lieutenant  General.  USAF 
Program  Executive  Officer 


VWJ^aojUULD 

WENDY  Mv  MASIELLO 


Lieutenant  General  USAF 
Defense  Contract  Management  Agency 


Enclosures: 

I ,  F-35  JPO  and  DCMA  Response  to  Findings/ Recommendations 


cc: 

Office  of  the  Under  Secretary  of  Defense  Tor  Acquisition.  Technology,  and  Logistics 
Assistant  Secretary  of  the  Navy  for  Research,  Development  and  Acquisition 
Assistant  Secretary  o  f  the  Air  Force  (Acquisition) 
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Management  Agency  Comments  (cont'd) 


remornciALUsr  only 

Combined  F-35  JPO  and  DCMA  Responses  to  Do D  IG 
F-35  Engine  Quality  Assurance  Inspection  Report 
Project  No.  D2014-DTQTAD40Q3MO 

Finding  A 

Additional  Program  Management  Oversight  Required 

We  found  that  additional  program  management  oversight  is  required  by  F-35  JPO  and  DCMA,  as 
evidenced  by  the  61  nonconformities  (violations  of  AS9100C,  regulatory  requirements,  and  DoD  policy) 
that  we  documented  during  our  inspection.  F*35  JPO  and  DCMA  oversight  is  required  to  prevent  adverse 
program  cost  schedule,  and  performance  issues. 

JPO  Response:  Disagree  The  JPO  believes  that  adequate  program  management  oversight  t$  in  place, 
However,  specific  issues  wilt  be  addressed  as  the  JPO  deems  necessary  to  prevent  adverse  program 
cost,  schedule,  and  performance  issues. 

Finding  A  Discussion 

The  F  35  JPO  awarded  SDD  and  LRlP  contracts  with  CDRL  items  lhat  were  delivered  for  Information  only, 
and  not  for  approval  We  also  found  that  DCMA  was  not  notifying  the  procuring  contracting  officer  of  their 
absence  in  the  distribution  fist  for  CDRLs  impacting  program  surveillance  activities.  The  absence  of  F-3S 
JPO  approval  and  DCMA  review  of  GORL  items  limits  the  ability  of  the  government  to  provide  clear 
direction  and  guidance  to  the  contractor 

DCMA  Response:  Agree.  DCMA  A  PQ- Pratt  &  Whitney  did  not  re  vie  w  ail  CDRLs  to  identify  deliverables 
that  impact  DCMA  program  surveillance.  DCMA  believes  this  nsk  was  mitigated  based  on  access  to  Pratt 
&  Whitney's  electronic  document  storage  System  where  many  of  the  GORL  items  can  be  found  and 
utilized  for  program  surveillance  Going  forward,  DCMA  APO-PraU  &  Whitney  wilt  update  Contract  receipt 
and  review  process  checklists  to  ensure  DCMA  is  included  in  the  distribution  of  aii  CDRLs, 

The  F-35  JPO  used  the  variance  process  to  waive  requirements  for  extended  periods  of  time,  which  was  a 
misapplication  of  the  variance  request  process  and  did  not  discourage  the  repeated  tender  of 
nonconforming  supplies  or  services  in  accordance  with  FAR  46.407,  "Nonconforming  Supplies  or 
Services."  To  date,  there  are  45  outstanding  MVRs  of  the  original  175  MVRs  from  the  production  baseline 
in  2009;  approximately  37  MVRs  could  lead  to  specification  changes  to  meet  the  performance  limitations. 
The  F-35  JPG  will  continue  to  accept  nonconforming  engines  if  MVR  extensions  continue  to  be  approved 
and  not  resolved. 

DCMA  could  not  provide  evidence  of  evaluating  and  making  recommendations  for  the  acceptance  or 
rejection  of  waivers  and  deviations  According  to  FAR  42.302,  "Contract  Administration  Functions."  DCMA 
is  to  assist  In  evaluating  and  make  recommendations  for  acceptance  or  rejection  of  waivers  and  deviations. 
The  MOA  between  the  F  35  JPO  and  OCMA  did  not  relieve  DCMA  of  this  function,  DCMA  needs  to 
perform  this  function  to  ensure  that  variances  are  correctly  classified,  reviewed,  and  approved. 

DCMA  Response:  Partially  Agree  FAR  46.407(d)  states  If  the  nonconformance  is  minor ;  the  cognizant 
contract  administration  office  may  make  the  determination  to  accept  or  reject,  except  where  this  authority 
is  withheld  by  the  contracting  office  of  the  contracting  activity , *  The  FI  35  LRlP  6  and  7  contracts  grant 
Pratt  &  Whitney  Material  Review  Board  (MRS)  authority  for  disposition  of  minor  nonconformances  without 
government  approval  Since  the  contractor  has  this  authority  by  virtue  of  the  contract  language,  DCMA’s 
evaluation  and  recommendations  of  acceptance  or  rejection  of  waivers  or  deviations  as  stated  in  FAR 
42.302(a)  (47)  is  being  performed  as  stated  in  the  contract  w Minor  non-conformances  processed  through 
the  contractors  Material  Review  Board  (MRB)  are  subject  to  DCMA  post  review. "  The  FI 35  MOA  with  the 
JPO  further  states:  " DCMA  wifi  monitor  minor  waivers  at  the  Pratt  &  Whitney  Material  Review  Board 
(MRB)  by  surveillance  of  the  PW  MRB  activities,  if  allowed  in  the  contract.  If  DCMA  questions  the  clarity 
or  technical  content  of  a  MRB  action,  DCMA  will  request  PW  clarify  with  supporting  documentation  and 
DCMA  may  discuss  the  condition  with  the  JPG's  responsible  (PT  Lead.*  DCMA  APQ-Pratt  &  Whitney 
also  has  the  right  to  reject  engines  presented  at  final  inspection  to  the  Government  when  the  Government 
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F-35  Joint  Program  Office  and  Defense  Contract 
Management  Agency  Comments  (cont'd) 


rorcomciAi  ost  only 


finds  fault  with  their  MRS  determination  through  post- review.  DCMA  A PO Pratt  &  Whitney  wilt  review  and 
update  fiie  MQA  with  the  F-35  Joint  Program  Office  to  clarify  DCMA's  functions  for  evaluating  waivers 
and  deviations, 

DCMA  could  not  provide  sufficient  evidence  of  supporting  information  to  substantiate  closure  of  CARs. 
approval  of  contractor  corrective  action  plans  for  CARs,  verification  of  corrective  actions  for  CARS, 
follow-up  on  Level  If  CARsr  and  escalation  of  CARs  to  the  next  higher  level  when  corrective  actions  were 
deemed  ineffective.  Without  sufficient  evidence  of  adherence  to  the  corrective  action  process,  we  do  not 
have  confidence  that  deficiencies  were  captured,  completed,  and  verified  to  prevent  recurrence  of 
deficiencies  in  affected  products. 

DCMA  Response:  Partially  Agree.  White  DCMA  could  not  provide  ail  requested  corrective  action  data 
within  the  lime  frame  desired  by  the  DODIG,  documentation  supporting  CAR/CAP  approval,  closure  and 
verification  is  available,  DCMA  is  working  on  a  central  repository  to  maintain  corrective  action 
documentation  to  ensure  it  is  more  readily  accessible  and  retrievable  in  the  future.  DCMA  agrees  we  did 
not  execute  to  DCMA  Instruction  U12Q1  and  that  this  instruction  requires  clarification  It  is  currently  under 
revision  to  address  clarification  to  the  CAR  elevation  process  DCMA  Headquarters  wifi  provide  an 
updated  policy  with  an  estimated  completion  date  of  early  2015.  Upon  policy  update  approval,  DCMA 
APO-Pratt  6  Whitney  will  review  all  open  CARS  to  determine  if  any  warrant  elevation  to  the  next  level  as 
recommended,  DCMA  APO-Pratt  &  Whitney  will  assess  ell  open  CARs  based  on  new  policy  guidance  to 
identify,  analyze,  and  assess  any  impacts  to  F-35  product 

Recommendation  A.  1 

We  recommend  that  the  F-35  Joint  Program  Office: 

a}  Coordinate  with  the  Defense  Contract  Management  Agency  to  conduct  an  effective  root  cause  analysts 
and  implement  corrective  actions  for  all  61  nonconformities  { violations  of  AS910QC,  regulatory 
requirements,  and  DoD  policies)  identified  during  our  Inspection, 

JPQ  Response:  Agree  JPQ  wilt  coordinate  with  DCMA  to  conduct  an  effective  root  cause  analysis  and 
imple  ment  corrective  actions  for  ail  6 1  noncon  formities.  Comph  to  by  end  of  Dec  2015 

b)  Review  the  contract  data  requirements  list  and  determine  specific  items  that  should  require  approval 
JPQ  Response:  Agree,  JPQ  wifi  review  CDRLs  and  determine  specific  items  that  should  require 
approval.  The  JPQ  has  reviewed  CDRL  approval  determination  every  year  while  preparing  the  annual 
LRtP  RFPs.  LRfP  1 1  RFP  will  be  released  later  this  year  Complete  by  end  of  April  2015. 

c)  Evaluate  open  major  variance  requests  to  determine  whelher  specification  changes  are  required  and  if 
achievable  closure  plans  can  be  developed. 

JPO  Response:  Disagree,  The  JPO  variance  process  is  compliant  The  FI 35  Program  is  still  in 
development  and  is  working  to  aith&r  meet  or  Change  the  requirements  at  the  end  ofSDD.  AH  FI 35  MVRs 
have  achievable  closure  plans  in  accordance  with  the  F-35  acquisition  strategy. 

Recommendation  A.  2 

We  recommend  that  the  Defense  Contract  Management  Agency: 

a)  Review  the  contract  data  requirements  list  items  to  identify  any  deliverables  that  impact  Defense 
Contract  Management  Agency's  program  surveillance  and  coordinate  with  the  F-35  Joint  Program  Office 
to  resolve  contract  data  requirements  list  distribution  issues, 

DCMA  Response:  Agree.  DCMA  APO-Pratt  &  Whitney  wilt  ensure  Contract  Data  Requirements  Lists 
(CDRLs)  are  reviewed  as  pad  of  the  contract  receipt  and  review  process  to  ensure  CDRLs  required  for 
program  sun/eliiance  are  identified  and  coordinated  with  the  F-35  Joint  Program  Office  and  the  contractor 
to  ensure  proper  distribution  is  made  to  DCMA  APO-Pratt  &  Whitney,  Contract  receipt  and  review  process 
checklists  will  be  updated  to  ensure  DCMA  is  included  in  the  distribution  of  ait  CDRLs 

b)  Review  and  update  the  Memorandum  of  Agreement  with  the  F-35  Joint  Program  Office  to  ensure  that 
aJJ  Defense  Contract  Management  Agency  functions  to  evaluate  waivers  and  deviations  are  dear  and  in 
accordance  with  the  federal  Acquisition  Regulation. 
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DCMA  Response:  Agree.  DCMA  APO-Pratt  &  Whitney  will  review  and  update  the  MOA  With  the  F-35 
Joint  Program  Office  to  clarify  DCMA's  functions  for  evaluating  waivers  and  deviations  and  ensuring  they 
am  dear  and  ~m  accordance  with  the  Federal  Acquisition  Regulation. 


c)  Ensure  that  sit  associated  data  required  to  substantiate  corrective  action,  closure  is  accessible, 
available,  and  retrievable. 

DCMA  Response:  Partially  Agree,  While  DCMA  could  not  provide  all  requested  corrective  action  data 
within  the  time  frame  desired  by  the  DODIG ,  documentation  f$  available.  DCMA  is  working  an  a  central 
repository  to  maintain  corrective  action  documentation  to  ensure  it  is  more  madily  accessible,  available 
and  retrievable. 

Finding  B 

Critical  Safety  item  Program  Not  Fully  Compliant 

The  F135  CSl  program  did  not  meet  DoD  OS!  requirement  In  addition.  Pratt  &  Whitney  did  not  meet  CSl 
contractual  requirements.  A  CSl  program  that  is  not  fully  compliant  with  DoD  policy  inhibits  the  ability  of 
FI 35  personnel  responsive  for  design,  development  production,  test,  maintenance  and  inspection  to 
perform  effective  CSl  life  cycle  management. 

JPO  Response:  Partially  agree.  CSl  requirements  were  flawed  to  the  contractor  for  LRIP1  at  which  time 
the  F-35/F135  CSl  requirements  were  shit  being  formulated.  Beginning  with  LRIP  9  the  program  wilt 
include  the  pari  number  ateo.  The  JPQ  is  working  to  align  with  updated  DoD  and  department  instructions 

Recommendation  B.1 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  critical  safety  item  contractual 
requirements  and  Pratt  &  Whitney's  critical  safety  item  program  processes  and  specifications  meet  the 
intent  of  the  Joint  Critical  Safety  Item  Instruction  and  supplemental  guidance  of  the  Joint  Aeronautical 
Commanders  Group  Aviation  Critical  Safety  Item  Management  Handbook. 

JPO  Response:  Partially  agree  CSt  requirements  were  flowed  to  toe  contractor  forLRtPl  at  which  time 
the  F -35/FI 35  CSl  requirements  were  stilt  being  formulated.  FI 35  CSl  parts  are  identified  by 
nomenclature,  not  part  number  However  beginning  with  LRIP  9  the  program  w HI  include  the  pari  number 
also.  The  JPO  is  working  to  align  with  updated  DoD  and  department  instructions.  JPO  is  working  with  the 
contractor  to  document  the  process  to  allow  for  full  alignment  with  DOD  requirements.  Complete  by  May 
2015. 

Recommendation  B.2 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  critical  safety  items  for  already  delivered 
engines  meet  DoO  critical  safety  item  requirements  such  that  critical  safety  items  receive  proper  care  and 
handling  during  installation,  testing,  and  Reid  maintenance. 

JPO  Response:  Disagree  During  execution  of  Recommendation  of  B.1,  if  it  is  discovered  that  a 
significant  escape  has  occurred ,  then  the  JPO  wilt  take  appropriate  action  for  delivered  engines. 

Finding  C 

Contractor  Shortfalls  in  Continuous  Improvement 

The  F-35  JPO  did  not  establish  FI 35  program  quality  goals  and  objectives  feat  were  mutually  agreed 
upon  by  Pratt  &  Whitney  for  current  contracts.  Additionally,  Pratt  &  Whitney  metrics  did  nol  show 
improvement  in  quality,  process  capability,  and  WBtKKKMSM  if  the  FI 35  program  does  not  make 
continuous  improvement  a  lop  priority,  product  quality  could  degrade,  especially  with  increasing 
production  quantities  over  the  next  several  years 

JPQ  Response:  Partially  agree.  JPO  accepted  the  P&W  Company  Quality  Plan  in  lieu  of  a  specific  P&W 
FI 35  Quality  Plan,  There  is  no  Department  or  other  defined  requirement  for  a  program-specific  Quality 
Plan  nor  is  it  required  by  SOW.  The  Program  Manager  has  the  responsibility  to  decide  how  to  proceed 
based  upon  the  size/comphxityfrisk  of  the  program 

3 


50  |  DODIG-2015-111 


FOR  OFFICIAL  USE  ONLY 


FOR  OFFICIAL  USE  ONLY 


Management  Comments 


F-35  Joint  Program  Office  and  Defense  Contract 
Management  Agency  Comments  (cont'd) 


EOR  OFFICIAL  USE  ONLY 


Recommendation  C.1 

We  recommend  that  the  F-35  Joint  Program  Office  establish  FI 35  program  quality  goals  and  objectives 
that  are  mutually  agreed  upon  by  Pratt  &  Whitney  and  track  Pratt  &  Whitney1  s  performance  against  those 
objectives. 

JPO  Response:  Partially  agree.  Program  quality  goafs  and  objectives  have  been  established  and  agreed 
upon  They  are  tracked  and  presented  to  program  management.  Since  the  PEQs  November  2014  Quality 
Update  the  program  has  established  FI 35  specific  program  goals.  This  will  be  reflected  in  future 
production  contracts. 

Recommendation  C,2 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney  s  quality  plan  meets 
contractual  requirements 

JPO  Response:  Agree.  JPO  will  ensure  that.  Pratt  &  Whitney's  quality  plan  meats  contractual 
requirements  as  reflected  in  future  production  contracts. 

Recommendation  C3 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney  consistently  analyzes  and 
reports  KPC  Cpk  data  for  FI 35  engine  hardware  and  that  performance  improvement  plans  are 
established, 

JPO  Response:  Agree.  JPO  will  ensure  that  Pratt  &  Whitney  consistently  analyzes  and  reports  KPC  Cpk 
data  for  FI 35  engine  hardware  and  that  performance  improvement  plans  are  established.  Complete  by 
end  of  April  2015. 

Finding  D 

Insufficient  Risk  Management  Practices 

The  F  35  JPO  did  not  ensure  that  Pratt  &  Whitney  proactively  identify,  elevate,  hack,  and  manage  FI 35 
program  risks,  in  accordance  with  the  FI 35  risk  management  plan.  As  a  result,  risks  that  can  adversely 
affect  the  Ft 36  program  may  not  receive  program  management  visibility  and  awareness. 

JPO  Response;  Disagree.  Contactor  follows  standard  risk  management  processes  for  identification  and 
tracking  The  team  actively  work  identified  risks  High  level  risks  have  visibility  across  the  air  system 
management  teams  at  regular  meetings. 

Recommendation  Q 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney  identify,  elevate,  track, 
and  manage  all  risks  that  affect  the  program,  including  software  and  supply  chain  risks. 

JPO  Response:  Partially  agree.  JPO  will  ensure  that  Pratt  &  Whitney  identify,  elevate,  track,  and  manage 
all  risks  that  affect  the  pregram,  Including  software  and  supply  chain  risks.  The  FI 35  RiMB  is  not  the  only 
process  used  by  P&W  and  the  JPO  Software  system  level  risks  are  already  evaluated  and  documented 
at  the  software  integrated  product  ream  board  weekly  meeting.  Items  with  significant  air  system  level 
impact  will  also  recerve  an  Air  System  SPAR  Board  severity  assessment  Those  issues  with  high  criticality 
rank  which  impact  safety  or  major  program  objectives  are  treated  property  by  elevating  to  program 
management  and  taking  immediate  action  to  fix  the  problem. 

Finding  E 

Insufficient  Supplier  Selection  Criteria  and  Management  of  Underperforming 
Suppliers 

The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney's  supplier  selection  criteria  and  management  of 
underperforming  suppliers  were  sufficient.  As  a  result  Pratt  &  Whitney  cannot  provide  the  F-35  JPO 
assurance  that  their  suppliers  are  qualified  to  deliver  quality  products 
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JPO  Response:  Partially  agree,  JPO  disagrees  that  it  did  not  ensure  that  supplier  selection  was 
sufficient;  however  does  agree  to  re- evaluate  PW's  management  of  underperforming  suppliers  to  ensure 
suppliers  are  qualified  to  deliver  quality  products. 

Recommendation  EJ 

We  recommend  that  the  F-35  Joint  Program  Office  work  with  the  Defense  Contract  Management  Agency 
to  ensure  that  Pratt  &  Whitney  clearly  defines,  documents,  and  implements  minimum  baseline  criteria  for 
supplier  selection  and  actions  to  be  taken  for  suppliers  that  continue  to  be  high  risk. 

JPO  Response:  Agree.  JPO  wilt  worH  w/to  the  DCMA  to  ensure  that  Pratt  &  Whitney  clearly  defines, 
documents,  and  implements  minimum  baseline  criteria  tor  supplier  selection  and  actions  to  be  fatten  for 
suppliers  that  continue  to  be  high  risk. 

Recommendation  E2 

We  recommend  that  the  Defense  Contract  Management  Agency  perform  additional  surveillance  on  Pratt. 
&  Whitney's  corrective  action  requests  issued  to  suppliers  to  ensure  that  they  are  closed  within  a 
reasonable  timeframe. 

DCMA  Response:  Agree  DCMA  APO-Pratt  &  Whitney  will  Incorporate  supplier  corrective  action  into  its 
Quality  Management  System  (QMS)  surveillance  plan  to  ensure  Pratt  &  Whitney  management  of  supplier 
corrective  action  is  In  compliance  with  their  quality  system 

(FQUO)  Finding  F 

Inadequate  Software  Quality  Management  Practices 

The  F-35  JPO  did  not  ensure  that  Pratt  &  Whitney  demonstrated  adequate  software  quality  management 
practices.  Pratt  &  Whitney  had  an  obsolete  SOP  and  had  requirements  that  wore  not  fully  implemented  or 
traceable.  Additionally,  Pratt  &  Whitney  was  not  performing  required  audits  and  reviews  to  ensure  that 
software  products  meet  contractual  requirements.  As  a  resuit,  there  is  no  assurance  that  software 
products  meet  contractual  requirements  and  there  is  increased  risk  to  performance  of  critical  software. 
JPO  Response:  Disagree.  Software  quality  management  practices  are  in  place  and  technical  reviews 
have  bean  conducted.  SDR  was  in  review  for  updates  prior  to  the  IG  audit 

fteco/nmendafton  F,i 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney  resolve  nonconformities 
related  to  software  quality  management  systems,  Including  an  obsolete  FI 35  Software  Development 
Plan,  lack  of  software  requirements  traceability,  and  software  not  developed  to  the  appropriate  product 
safely  level. 

JPO  Response:  Disagree.  Software  is  developed  to  the  correct  product  software  level.  The  software 
development  plan  is  not  obsolete  but  only  needs  some  minor  updates  that  were  already  in  work  prior  to 
the  IG  Audit.  These  updates  do  not  Invalidate  the  software  flight  clearances  completed  to  date  as  they 
merely  clarify  existing  processes  and  procedures. 

Recommendation  F.2 

We  recommend  the!  the  F-35  Joint  Program  Office  ensure  that  Pratt  &  Whitney  s  Software  Quality 
Assurance  organization  conduct  audits,  reviews,  and  verification  activities  of  both  internally  developed 
and  supplier-developed  software. 

JPO  Response:  Agree  JPQ  will  ensure  that  Pratt  &  Whitney's  Software  Quality  Assurance  organization 
conduct  audits,  reviews,  and  verification  activities  of  both  internally  developed  and  supptter-devetopad 
software. 

Recommendation  F.3 

We  recommend  that  the  F-35  Joint  Program  Office  ensure  Pratt  &  Whitney  assess  the  impact  of 
Insufficient  software  verification  on  delivered  engines. 

JPO  Response:  Disagree  The  F135  propulsion  system  software  was  properly  tested  against  the 
standards  called  out  in  the  software  development  plan  and  standard  work  Multiple  flight  clearances 
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performed  strwe  the  beginning  of  the  FI  35  program  are  supported  by  the  excellent  safety  record  for 
software  in  engine  ground  test,  flight  test  and  production 
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APO  Aircraft  Propulsion  Ops 
CAR  Corrective  Action  Request 
CDRL  Contract  Data  Requirements  List 
CMM  Coordinate  Measurement  Machine 


Cpk  Process  Capability  Index 
CSCI  Computer  Software  Configuration  Item 
CSI  Critical  Safety  Item 
CTOL  Conventional  Takeoff  and  Landing 
CV  Carrier-Suitable  Variant 
DCMA  Defense  Contract  Management  Agency 
ESDS  Electrostatic  Discharge  Sensitive 
EVB  Electronics  Verification  Bench 
FAR  Federal  Acquisition  Regulation 
FADEC  Full  Authority  Digital  Engine  Control 
FCA  Functional  Configuration  Audit 
FOD  Foreign  Object  Damage 
FRP  Full-Rate  Production 
GCQA  Government  Contract  Quality  Assurance 
IMP  Integrated  Master  Plan 
IMS  Integrated  Master  Schedule 
JACG  Joint  Aeronautical  Commanders  Group 
JPO  Joint  Program  Office 
KPC  Key  Product  Characteristics 
LRIP  Low-Rate  Initial  Production 
MOA  Memorandum  of  Agreement 
MPR  Monthly  Program  Review 
MVR  Major  Variance  Request 
NDT  Non  Destructive  Test 
OFI  Opportunity  for  Improvement 
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PCA  Physical  Configuration  Audit 
PHM  Prognostics  and  Health  Management 
PPE  Personal  Protective  Equipment 
PSDP  Propulsion  System  Development  Plan 
PSL  Product  Software  Level 
RIMB  Risk  and  Issues  Management  Board 
SDD  System  Development  and  Demonstration 
SDP  Software  Development  Plan 
SLC  Software  Life  Cycle 
SPAR  Software  Problem  Anomaly  Report 
SOW  Statement  of  Work 
SQA  Software  Quality  Assurance 
STOVL  Short  Takeoff  and  Vertical  Landing 
UTAS  UTC  Aerospace  Systems 
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Whistleblower  Protection 

U.S.  Department  of  Defense 

The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report@listserve.com 

Twitter 

twitter.com/DoD_IG 

DoD  Hotline 

dodig.mil/hotline 
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